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ABSTRACT 


In this thesis, we present a software paekage update faeilitating the evaluation of 
the Systems Engineering Value Equation with Risk (SEVER) using the TEMPO Military 
Planning Game originally developed by General Eleetrie in the 1960s. Currently, it is 
used by Naval Postgraduate Sehool to train future military deeision makers in making 
eritieal resource allocation decisions. The intent of this development facilitates creating a 
software decision tool adaptable to different software-based resource allocation models 
such that decision makers are presented with customized, relevant information for both 
‘satisfaction’ and ‘optimization’ decisions along with a probability of successful outcome 
prior to executing the decision and observing the effects. The software package 
development (identified as TEMPO Version 3) involved porting software code from C++ 
into Visual Basic.NET, integrating MATLAB code for numerical strategy execution, and 
allowing for future strategy generation and statistical analysis to compare against 
SEVER-predicted outcomes. SEVER is the algorithm proposed by Eangford and Hyuhn 
(2006) that facilitates Systems Engineering decision making. TEMPO Version 3 is the 
particular software application proposed to evaluate the SEVER algorithm once it is 
developed into a software package. SEVER was not implemented; however, it was 
considered during the development of the updated software package. 
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I. 


INTRODUCTION 


A. MOTIVATION 

In light of today’s fast-paced and dynamic global environment, it is imperative for 
United States Department of Defense (DoD) decision makers to evaluate and execute 
critical strategic decisions efficiently. Among the most important of these decisions are 
managing current assets, funding development of promising future technologies, 
converting state-of-the-art technologies into war-fighting applications, and inserting 
Commercial-off-the-Shelf (COTS) technology into military systems. Understanding the 
factors involved in making these decisions can be complex and convoluted. Simplifying 
and clarifying the interactions between these factors would benefit decision makers at 
every level. Decision support systems exist within the DoD to guide the decision maker 
in making a proper decision. 

According to the Defense Acquisition Guidebook published in 2003 (updated in 
2008), the DoD has three principal decision support systems in place. These are the 
Planning, Programming, Budgeting and Execution (PPBE) Process, Joint Capability 
Integration and Development System (JCIDS), and the Defense Acquisition System. The 
PPBE process is used to craft plans and programs that satisfy the demands of the National 
Security Strategy within resource constraints. The JCIDS provides a systematic method 
established by the Joint Chiefs of Staff for assessing gaps in military joint warfighting 
capabilities and recommending solutions to resolve these gaps. The Defense Acquisition 
System is a management process by which the DoD acquires weapon systems and 
automated information systems. Together, the three systems provide an integrated 
approach to strategic planning, identification of needs for military capabilities, systems 
acquisition, and program and budget development. 

Decisions are frequently outcomes of trade studies. A trade study, or trade-off 

study, is the activity of a multidisciplinary team to identify the most balanced technical 

solutions among a set of proposed viable solutions (Eederal Aviation Administration 

[EAA], 2006). Identifying ‘the most balanced technical solution’ involves clearly 
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understanding the set of multiple, competing criteria and subsequently ranking their 
importance. For the purposes of this research, decisions and trade studies are considered 
synonymous. Both decisions and trade studies involve evaluating the pros and cons, or 
risks and rewards of possible outcomes. If all options constituting a decision’s outcome 
could be analyzed thoroughly (e.g., through some inductive or deductive process), such 
that the risks and rewards for each were objectively defined and understood, we surmise 
that the possible outcomes of a decision could be evaluated objectively. And, just as 
importantly, if this risk/reward analysis could be automated for a given situation, perhaps 
the time to make that decision could be reduced. Then the focus would center on 
increasing a decision maker’s effectiveness in evaluating the possible outcomes. By 
assessing the likelihood and consequences of outcomes and decreasing the time to make a 
decision, the risks of making the wrong decision at the wrong time could be diminished. 

However, if information on the interacting factors that confound a decision is 
incomplete or unorganized, the limited time in which to make a decision could be wasted 
through physically and mentally organizing the information. Disorganization could result 
in inaccurate assessment of an outcome’s consequence. Coupling today’s powerful 
computational processing with a new, innovative methodology based on a total systems 
perspective to organizing information, the author posited that a decision evaluation tool 
which bridges the gap between ‘satisfaction decisions’ and ‘optimization decisions’ could 
be developed to help increase performance and quality of a decision maker’s execution. 

The research and development presented in this thesis developed the groundwork 
for incorporating and testing an algorithm that is the core of such a software tool. This 
algorithm is currently known as the Systems Engineering Value Equation with Risk 
(SEVER), and was originally derived by Gary Eangford in 2006. 

B. BACKGROUND 

“Decision” is a common term used in the English language covering a variety of 
topics. In sports, a decision means a win or loss. In law, a decision determines a person’s 
guilt or innocence. Regarding this thesis, a decision is a position or opinion or judgment 
reached after consideration (American Heritage Dictionary, 2004). Every day, people 
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must make decisions. The impacts of these decisions directly reflect the amount of time 
required to spend considering outcomes prior to making that decision, or deciding. There 
are many areas of research relating to analyzing and forecasting a decision’s potential 
impact or outcome. If the decision warrants the effort, decision theory can be applied. 
Resnik (1987) states: 

Decision theory is the product of the joint efforts of economists, 
mathematicians, philosophers, social scientists, and statisticians toward 
making sense of how individuals and groups make or should make 
decisions. The applications of decision theory range from very abstract 
speculations by philosophers about ideal rational agents to practical advice 
from decision analysts trained in business schools....It is thus usual to 
divide decision theory into two main branches: normative (or prescriptive) 
decision theory and descriptive decision theory. Descriptive decision 
theorists seek to find out how decisions are made—they investigate 
ordinary mortals; their colleagues in normative decision theory are 
supposed to prescribe how decisions ought to be made—they study ideally 
rational agents. 

This thesis focused on normative decision theory. Further, two classes of 
normative decisions were considered: ‘satisfaction’ and ‘optimization’ decisions. Let a 
satisfaction decision be defined as achieving the highest expected utility under a given 
situation. According to Bernoulli (1738), the determination of the value of an item must 
not be based on its price, but rather on the utility it yields. The price of the item is 
dependent only on the thing itself and is equal for everyone; the utility, however, is 
dependent on the particular circumstances of the person making the estimate. Thus, the 
best outcome of a satisfaction decision is one that reflects the highest expected utility. 

In the other decision type, ‘optimization,’ decisions are based on optimization 
theory. According to Jongen et al. (2004), optimization theory is the mathematical study 
of problems that ask for minimal or maximal values of an objective function on a given 
domain. Considering this definition, let an optimization decision be defined as finding the 
minimum or maximum values (i.e., maximizing or minimizing) for a pre-defined, 
representative mathematical model of the situation (i.e., the objective function). Further, 
the subject of multiple criteria optimization is the selection of good decisions from a set 
of alternatives with respect to multiple criteria or objective functions (Steuer, 1986). 
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These types of decisions usually revolve around conflicting objective functions in which 
Pareto-efficient solutions exist. In general, Pareto efficiency, or Pareto optimality, is 
easily described with respect to a game; An outcome of a game is Pareto efficient if there 
is no other outcome that makes every player at least as well off and at least one player 
strictly better off (Gametheory.net, n.d.). In the case of multiple criteria optimization, the 
solution is Pareto efficient if there is no solution for which a particular criterion’s solution 
can be improved without simultaneously degrading another. At the point of Pareto- 
efficiency, the decision maker’s discretion is required to make the ultimate decision, as 
the existing Pareto-solutions are independently optimal against the conflicting objective 
functions. The methods used to solve multiple objective optimization problems include 
Markov Decision Processes (Bellman, 1957) through dynamic programming and 
reinforcement learning (Sutton & Barto, 1998), multi-attribute utility theory (MAUT) 
(Keeney & Raiffa, 1976), and evolutionary algorithms (Abraham, Jain, & Goldberg, 
2005), to name a few. Each of these methods requires the situation to be represented in 
the form of a mathematical relationship, or objective equation, in order to execute these 
numerical methods. A quick summary of each follows. 

According to Howard (1960): 

A Markov Process is a mathematical model that is useful in the study of 
complex systems. The basic concepts of the Markov process are those of 
“state” of a system and state “transition.”...A graphic example of a 
Markov process is presented by a frog in a lily pond. As time goes by, the 
frog jumps from one lily pad to another according to his whim of the 
moment. The state of the system is the number of the pad currently 
occupied by the frog; the state transition is of course his leap. ...If we 
focus our attention on the state transitions of the system and merely index 
the transitions in time, then we may profitably think of the system as a 
discrete-time process....To study the discrete-time process, we must 
specify the probabilistic nature of the state transition. 

This method is difficult to implement without accurate knowledge of the 
probabilistic nature state transitions. Many decisions are not consistently encountered, or 
have previously been statistically modeled. 
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According to Sutton and Barto (1998), reinforcement learning is learning what to 
do—how to map situations to notions—so as to maximize a numerical reward signal. The 
learner is not told which actions to take, as in most forms of machine learning, but instead 
must disoover whioh actions yield the most reward by trying them. 

MAUT is a special case of the Unidimensional Utility Theory involving more 
than one utility function (Keeney & Raiffa, 1976). Keeney and Raiffa (1976) summarize 
Unidimensional Utility Theory as follows: 

A decision maker must ohoose among several alternatives... eaoh of whioh 
will eventually result in a consequenoe describable in terms of a single 
attribute. The deoision maker does not know exaotly what consequence 
will result from eaoh of the various alternatives, but he can assign 
probabilities to the various possibilities that might result from any course 
of action....If an appropriate utility is assigned to eaoh possible 
oonsequence and the expected utility of each alternative is calculated, then 
the best oourse of aotion is the alternative with the highest expected utility. 

The differenoe between unidimensional utility theory and MAUT is contained 
within the number of attributes against whioh the deoision maker must evaluate a 
deoision’s outcome. MAUT usually involves multiple optimum solutions, since in most 
oases the optimization of eaoh individual attribute does not lead to the optimization of all 
considered attributes. Performing trade studies and considering personal preferences are 
neoessary to evaluate which decision is best onoe the Pareto-effioient solutions are 
oomputed. 

Krantz and Kunreuther (2007) developed a subjeotive expeoted multi-attribute 
utility theory (SEMAUT), based on the determination that goals are deemed stable, and 
therefore the tradeoffs among the different goals can be approximately represented by a 
multi-attribute utility funetion. In the ease of TEMPO, two eonstructs distinguish it from 
the commonly held notion that MAUT has not been assoeiated with preseriptive 
approaches 

1. The notion that preferenees are revealed rather than eonstructed is an 
important distinction. SEMAUT’s frame is in sharp contrast to the idea 
that preferences are constructed rather than revealed (Tversky, Sattath & 
Slovic (1988); Tversky, Slovic & Kahneman (1990); Chapman & Johnson 
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(1995); Slovic (1995)). In the prescriptive paradigm, one concentrates on 
improving the choice process for a particular context given an 
understanding of how the computer behaves. Prescriptions should be 
formed on rules or norms which are constructed rather than designed 
through preference. SEMAUT focuses on values or utilities and decision 
weights rather than subjective probabilities, and 

2. The notion that there is a multitude of frameworks underscores 
prescriptive thinking. One must consider the correct temporal framing, 
how to commit resources to meet a goal, how should the decision¬ 
weighting process be enacted, and how to set the goals correctly in the 
context of the previous play. 

Evolutionary algorithms are based on the following three methods: genetic 
algorithms, evolution strategies, and evolutionary programming (Baech et ah, 2000). In 
general, Baech et al. (2000) explain evolutionary algorithms as: 

[Those that] utilize the collective learning process of a population of 
individuals.! Usually, each individual represents a search point in the 
space of potential solutions to a given problem. Additionally, individuals 
may also incorporate further information; for example, strategy parameters 
of the evolutionary algorithm. 

Each of these methods is mentioned as a potential way to solve certain types of 
problems involving decision outcomes. The intent of this discussion is to provide 
background information on other methods available, not on the details of their 
effectiveness or whether one method is preferred. 

Regarding ‘satisfaction decisions,’ a proper framework is required to understand 
the decision at hand in better detail (i.e., the information is presented in a hierarchical 
fashion to facilitate comparison between the alternatives) so that an educated decision can 
be made. These methods propose to decompose the decision such that the decision maker 
has a more objective dataset in order to make the most satisfying choice. Analytic 
network/hierarchy processes (ANP/AHP), both proposed by Saaty in 2001, and decision 
trees are two methods that establish a hierarchical view of a particular situation. 


! The term ‘individual’ refers to a single member eontaining a ehromosome or genome that usually 
eontains at least a representation of a possible solution to the problem being taekled. Other information 
sueh as eertain strategy parameters and the individual’s fitness value are usually also stored in eaeh 
individual. 
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Generally, two perspeetives exist in solving problems: “top-down” and “bottom- 
up” (Lakatos, 1978). Aeeording to Blanchard and Fabryky (2006). A top-down approach 
views the system as a whole, understanding how the system components effectively 
perform together. In contrast, a bottom-up approach refers to viewing a system by 
focusing on the lowest level system components and assembling upward. In general. 
Decision Theory methods such as the Markov Decision Process and MAUT rely on a 
bottom-up approach. That is, they require a numerical/mathematical model with assigned 
probabilities of the situation in order to compute the optimal outcome. These methods 
are concerned with solving an optimization decision problem. In cases based on laws of 
physics or proven empirical relationships, those methods work because software can be 
used to perform the intense computations. In these cases, parameter relationships through 
equations are derived, random inputs are provided, and computations or simulations are 
performed to provide the absolute range of possible outcomes as a probability 
distribution. The solution with the highest expected value, or utility, is considered the 
best. These models account for uncertain conditions by tuning particular parameters to 
calculate a new cumulative probability distribution for each case. In this sense, lower 
level model parameters are updated and give rise to a new system-level outcome. 

Many other methods, one being ANP/AHP, are more robust because they are 
typically applied in a top-down manner. The importance of each parameter in a decision 
varies from person to person. These methods allow the decision maker to provide custom 
criteria importance values in the form of criteria weighting, which more accurately 
represents the human behavioral aspect of decision making. 

SEVER is an algorithm that is most useful when applied in a top-down manner, 
yet is built on a bottom-up framework. SEVER is both an algorithm and a mathematical 
method used to capture and decompose a situation, or system, down to its lowest level 
elements. In fact, it is a Systems Engineering (SE), top-down, approach that provides a 
system-level view of the implications of executing a certain decision from the basic 
elements upward. Because of its SE approach, uncertainty is considered an input to 
SEVER in the form of risk. With respect to ‘satisfaction decisions,’ SEVER provides a 
decomposition framework in which to view a particular situation. With respect to 
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‘optimization decisions,’ SEVER provides the mathematical framework for comparing 
different potential outcomes so the optimal outcome for that situation can be discovered. 

One particular application of interest to the DoD involves training future program 
managers with the skill of allocating resources efficiently. A game has existed since the 
1960s to train students at the Naval Postgraduate School in this skill. It was originally 
developed by General Electric’s TEMPO Think Tank as a military budget allocation 
game and is known as the TEMPO Military Planning Game. TEMPO is a two-player 
game consisting of allocating finances on certain available weapon systems. Each 
weapon system provides a different level of military performance for a defined cost. Each 
player’s objective is to win a war based on the operating performance of particular 
inventoried weapons. The game has been modified from its original, paper-based version 
to a computer command console version to a spreadsheet-based version to its current 
Microsoft Windows form-based version. Through these evolutions, updates have been 
made to the game; however, the general intent of the game has remained the same just as 
has the skill it allows its users to hone: Allocate finances for particular weapon systems in 
order to win a war against your enemy. 

TEMPO was used to implement the constructs to support SEVER. TEMPO was 
modified to allow real-time feedback on allocation decisions made by the user. Once 
real-time feedback is integrated, SEVER will be capable of scoring the effectiveness of a 
decision in real time if the model. Many different strategies can be used during game 
play; SEVER was the tool used to analyze, or score, each strategy. 

C. PURPOSE 

This document describes the adaptation and update of the TEMPO Military 
Planning Game, developed by General Electric in the 1960s, to automatically execute a 
set of rule-based strategies. Eurther, a trial set of 1000 runs was executed for each 
strategy. The results of these trial sets are provided. TEMPO was updated to lay the 
groundwork for the future implementation of SEVER. Also, this work attempted to 
improve the TEMPO user interface. 
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By implementing rule-based strategies against the eomputer’s eoevolutionary 
rule-based strategies and recording the outcomes, evaluation of the main tool of the Value 
Systems Engineering process, SEVER, will be possible (Langford, 2006). Eor the 
formative purposes of this thesis, five major strategy types have been designed and 
implemented; 1. allocate randomly, 2. allocate based on reported probability of war, 3. 
allocated based on user-defined priority, 4. allocate based on offense or defensive 
preference, 5. allocate based on intelligence received. Of these, 10 particular (subset) 
strategies were tested. These strategies are subsets of the strategy types from changing 
certain parameters. These 10 strategies were executed and their comparative results are 
described. The methods used to compare these strategies serve as the foundation to 
evaluate SEVER. 

In summary, the work presented in this thesis creates a model on which to 
perform resource allocation trials based on strategies so that a statistical evaluation of the 
SEVER algorithm is possible. The TEMPO Military Planning Game (TEMPO) is the 
model selected to facilitate the evaluation of SEVER. MATLAB was used to create the 
strategy and analysis capabilities. The entire development is integrated as one software 
package that is user installed and executed. Erom this point forward, this software 
development is referred to as the ‘TEMPO Version 3.0.’ This software package can 
continue to be updated to include SEVER, when appropriate. 

The particular research question on which the work presented in this thesis is 
focused is: Can TEMPO be updated and modified to accommodate SEVER? What is 
required? How can this be achieved? 

D. SCOPE 

This thesis provides a development description of a decision model capable of 
both human (manual mode) and computer (automatic mode) execution in order to 
accommodate SEVER. The model (TEMPO) is designed such that a large number of 
trials (or ‘games’) can be executed, and the results recorded, to evaluate the player’s 
performance against the computer. The assimilation of SEVER and its evaluation is 
planned as a follow-on effort and is not discussed in this write-up. 
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TEMPO updates provide; 

• a meehanism to run a user-defined number of games with various user- 
seleeted rule-based strategies 

• a means to implement the SEVER evaluation algorithm via a Graphieal 
User Interfaee (GUI) 

• an enhaneed the user interfaee 

• a means to graphieally present the previous turn’s deeision results 

This thesis also presents the development and performance of 10 particular 
strategies autonomously played (i.e., computer versus computer) in order to validate 
TEMPO Version 3. Performance of each strategy is presented statistically. 

The scope of the work performed and presented concludes with the ability of 
TEMPO to record and present performance results of human player and computer 
automated strategies. This thesis does not present results comparing the actual 
performance results of these strategies against those predicted by SEVER—only results 
of the 10 currently coded strategies against each other. The strategy performance results 
are recorded by automatically executing a select number of trials of a particular user- 
selected strategy and saved within Extensible Markup Eanguage (XML) log files. 

A software development effort was necessary to update the existing TEMPO 
model from a grid-style interface to a more user-friendly Graphical User Interface (GUI). 
This development also included preliminary placeholder code so that SEVER can be 
implemented on the GUI along with special functions (coded in MATLAB) to analyze 
and present resource allocation results, where appropriate. Special attention was paid to 
the user interface to facilitate quick learning and reduce the effort required to understand 
and play the game. Previously, TEMPO was not capable of automatically playing a set 
number of games. During this thesis work, the software was modified and recoded to 
collect data for a particular user-selected strategy for a user-specified number of games. 
The updated software is capable of comparing actual performance of all types of logical 
rule-based strategies in addition to a player’s strategy. Eurther, a software framework to 
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develop and execute additional strategies in MATLAB was designed. The updated GUI 
allows human research on executing a strategy under time constraints to be possible, both 
with and without SEVER. 

E. FUTURE WORK 

Plans for future work include updating the preliminary placeholder code with the 
actual SEVER engine and applying the SEVER implementation to other numerical-based 
models. Particular future work for TEMPO and SEVER implementation will build upon 
the current strategies with new rule-based strategies, create a graphical user interface to 
streamline the post-processing and analysis of game log files, and perform a formal 
statistical analysis using the MATLAB Statistical Toolbox functions. 

Euture work regarding other models useful for SEVER could be a project 
schedule modeled in Microsoft Project© software where both financial and temporal 
decisions are required to successfully complete a project within given management 
time/budget constraints. The model would be of the software schedule created in 
Microsoft Project, and exported in an XML file format. Inputs to the SEVER algorithm 
include decisions regarding the tasks to perform versus the budget and time available. 
The output of the algorithm would be the risk, as a percentage, that scheduled project 
would not be successfully completed within the estimated timeframe and budget. 

F. DOCUMENT ORGANIZATION 

Chapter I contains the motivation and background necessary to understand the 
work presented within the thesis. 

Chapter II contains the background on both TEMPO and SEVER necessary for 
the reader to understand the subsequent software development effort. 

Chapter III contains two subsections. The first section presents the planned 
software development in order to answer the research question presented. The second 
section presents the statistical method used to evaluate the performances of each strategy. 
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Chapter IV describes the software package development, the methods involved in 
the strategy performance verification, and results of implementing each strategy into 
TEMPO. Further, it summarizes the software package development and draws 
conclusions from development of TEMPO’S updates and the rule-based strategies to 
answer the proposed research questions in Chapter I. Also, future development work and 
research based on the limitations within this thesis are suggested. 

The appendices provide more detailed information about TEMPO, SEVER, 
methods to integrate them, a description of the developed code (i.e.. Visual Basic.NET 
and MATEAB), software installation procedures, and game play instructions and 
features. A CD is packaged with this publication and contains all the software required to 
reproduce these results. 
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II. MODEL DESCRIPTIONS 


In order to understand the purpose and results presented within this thesis, the 
reader requires a suffieient understanding of the model and algorithm used. This ehapter 
is broken into two major seetions. The first seetion focuses on the description and game- 
play rules for TEMPO. The second section provides an overview of the SEVER 
algorithm and the method of applying this algorithm to TEMPO. 

A. TEMPO MILITARY PLANNING GAME 

According to Johnson et ah: 

In the early 1960s, the Department of Defense created a management 
system, the Planning, Programming, and Budgeting System (PPBS) of 
considerable complexity to rationalize its resource allocation problems 
(during a short range timeframe (1-5 years). A major training program 
was instituted to teach the PPBS and a “game” was created by General 
Electric’s “TEMPO Think Tank” to train people in the use of the new 
system. (2005) 

This training program comes in the form of a symmetric two-sided game. Teams 
of players compete in allocating limited resources to build up force structures consisting 
of notional weapon systems. The game consists of a number of turns, each turn 
representing one year. At the start of each “year,” each side receives a budget, which it 
may allocate to acquisition and operation of various weapon systems and to other 
categories such as “intelligence” and “counterintelligence.” Each year, conflict with the 
enemy becomes more probable, until a “war” occurs. At this point, each player’s force 
structure is matched against the opponent’s. Based on certain game rules, the more 
effective allocation win s . 

The TEMPO Military Planning game is an important training tool, enabling future 
military decision makers to simulate making critical resource allocation decisions within 
an uncertain finite timeline. In courses given by the Defense Resource Management 
Institute for more than 40 years, more than 20,000 students from 125 countries have 
played the game as part of their training. 
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The research reported by Johnson et al. (2005) was aimed at applying computing 
power to resource allocation in a competitive environment. They undertook to develop a 
computer program capable of playing the TEMPO game (in a somewhat simplified 
version) against human opponents. By using a coevolutionary algorithm, they were able 
to create such a program based on a small set of fuzzy-logic decision rules. According to 
Johnson et al. (2005), coevolutionary algorithms consist of individuals against other 
individuals, each competing for resources in an environment that, in itself, poses its own 
threats. Competing individuals use random variation and selection to seek out survival 
strategies that will give them an edge over their opposition. (Johnson et al. 2005). These 
survival strategies were formulated on the principles of fuzzy logic, originally introduced 
by Zadeh (1965) through fuzzy set theory. According to Zadeh (1965), more often than 
not, the classes of objects found in the real physical world do not have precisely defined 
criteria of membership...clearly, “the class of beautiful women,” or “the class of tall 
men”...are imprecisely defined classes. The imprecise classes provide for a framework 
that uses imprecise criteria to make decisions. 

Johnson et al. (2005) claim near human-level performance for the program on the 
grounds that it was able to beat one of its developers on the first several tries. In what 
follows, TEMPO refers to this computerized version of the game. 

1. Description (from “TEMPO Military Planning Game, Explanation 
and Rules for Players,” p. 15-21) 

a. Objective 

Player wins a war by having more Total Net Offensive (TNO) utils (see 
below for description) than the opponent. War can occur in any given year with 
increasing probability of occurrence each year the game continues. TNO utils are only 
calculated if war occurs. 
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b. Overview 

Weapon systems are grouped as follows: 

• Type: Offensive (O) or Defensive (D) 

• Class: A or B 

• Level: 1, 2, 4, or 4 

Weapon systems ean take any eombination of the above three levels of 
elassifioation. For example, OA2, DBS, ete., are eaeh valid weapon systems. Eaeh 
weapon system produees an output of military eapability that is valued in “UTILS.” The 
UTIL is a measure of effeetiveness used to simplify game playing and seoring. Although 
determining the effeetiveness of weapons is often the most diffieult part of military 
planning, this simplifioation permits the player to eoneentrate on the budget alloeation 
problem. The player’s goal is to allocate each weapon type so when war occurs they have 
more TNO utils than the opponent. Details on how this is accomplished follow. 

There are four categories of actions the player can execute for each turn: 

• ACQUIRE new weapon systems 

• OPERATE existing inventory of weapon systems 

• Purchase INTEEligence 

• Purchase COUNTER-INTEEligence 

An execution stage follows calculating and implementing these four 
decisions. The “execution stage” is defined as confirming all currently allocated decisions 
displayed on the screen and continuing to the next turn (i.e., next year). This execution 
stage is controlled by pressing the ‘Commit’ button. 

(1) Operate. This function is how military performance is 
calculated (in the form of UTILS). Operating existing inventory results in a number of 
UTILS multiplied by the number of units within inventory operated. There are penalties 
for operating too much of any single system, in the form of diminishing returns, which 
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are discussed later. By operating a particular weapon, the inventory for that weapon 
remains for the next year. This is one method to ensure there are enough units for next 
year. 

(2) Acquire. In order to OPERATE, units must exist in inventory. 
This is the other method to ensure there are enough units contained within inventory for 
the next year. Each year, a fixed amount of all available weapons are available to acquire. 

(3) General. Only weapons existing in the inventory in the current 
year can be operated. Therefore, if the player decides not to operate any units in a given 
year, only those weapons the player acquires that year will be available within the 
inventory the next year. All other pre-existing weapon inventories are lost. 

Only operating force units produce military capability and hence 
utils. New weapon systems do not necessarily replace existing systems; they might be 
additions to force structure. Eltil values for weapon systems remain constant throughout 
the game. Although util values per unit remain constant, if currently operated units in any 
one system produce more than 2000 utils, further utils are subject to “diminishing 
returns” on a sliding scale (see Eigure 1). Costs remain constant during procurement and 
operation of a given system. 
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VTE. .4I)JITST]MENTS TO REFLECT DDvIINISHING RETITRNS 
(FROM GROSS TO .ADRTSTED iniLS) 


GROSS UTILS (GU) 

ADJUSTED UTILS 

FROM 

TO 


1 

2000 

Same as Gross Utils 

2001 

3000 

2000 + .9 * (GU-2000) 

3001 

4000 

2900 - .8 • (GU-3000) 

4001 

5000 

3700 - .7 =" (GU-4000) 

5001 

6000 

4400 + .6 • (GU-5000) 

6001 

7000 

5000 - .5 * (GU-6000) 

7001 

8000 

5500 + .4 * (GU-7000) 

8001 

9000 

5900 + .3 • (GU-8000) 

9001 

10000 

6200 + .2 * (GU-9000) 

10001 

11000 

6400 + .1 * (GU-10000) 

11000 

rf. 

6500 



Figure 1. Implementation of Diminishing Returns (From; TEMPO Instruetions) 

Defensive weapon systems only defend against the opponent’s 
offensive weapon systems of the same Foree Unit Type; your DA weapons defend 
against only the enemy’s OA weapons, and your DB weapons defend only against the 
enemy’s OB weapons. These weapon systems eannot result in positive TNO utils. 
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Therefore, any DA system counts against any OA system of the enemy, and likewise for 
B systems, i.e., utils from operating DB2 defend against OBI, OB2, etc. However, once 
all OBI utils are defended, the extra utils are useless. In other words, no credit is given 
for “overdefending” in a Force Unit Type category. 


c. Scoring 


A team’s TNO utils for a particular year are calculated as follows: 


HUM 


( n n \ 

I HUM _ OA. - ^ COMP _ DA^ 

(=1 /=l / j 

f ^ ^ \ 

+ ^ HUM _ OB. - ^ COMP _ DB. 

V i^l J j 


( 2 . 1 ) 


COMP _ TNOujjj^g = X COMP _ OA. - ^ HUM _ DA. 

\/=l ' ,=l 

( n n 

Y, COMP _ OB. - Y hum _ DB. 


(=1 


/=1 


( 2 . 2 ) 


where prefixes HUM and COMP denote ‘human’ and ‘computer,’ respectively. 
TNOfjjj^^ J is the number of TNO Utils after year). (M, is the amount of utils derived 

from the type: offensive (O) weapon system class: A, level: i. OBi, DAi, and DBi are 
similarly defined. In this game, i ranges from 1-4. 


As mentioned previously, no credit is given for “overdefending” For 

n n 

example, if Y,HUM _DA. is greater than Y^OMP OA^ (both from first term in (2.2)) 

/=1 (=1 

there is no additional benefit. In mathematical terms: 


Y, hum _ DA^ 

i=i 


IS 


replaced with min s 'Y HUM _ DA , ^ COMP _ OA. 


(2.3) 


To win a war. 
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HUM 


TNO 


UTILS _ i 


> COMP 


j=WAR _ YEAR 


( 2 . 4 ) 


d. Game Play 

Normally, the game is played for several “years.” War is inevitable and 
will eventually “break out.” The amount of years before war oeeurs depends on a 
probability of war oeeurring. Eaeh year this probability inereases. 

e. Probability and Results of War 

The probability of war is announeed at the beginning of eaeh year and is 
for that year only. If war oeeurs, it only will happen after the budget deeisions for that 
year have been eompleted but before the next year begins. Neither team ean deelare a 
war. If war oeeurs, the results of (2.1) and (2.2) will be eomputed, and the eondition 
deseribed in (2.4) will be evaluated. At this point the game ends and a winner is deelared. 

f Budget 

Eaeh side has a finite budget. This budget inereases eaeh year by a random 
rate of inerease. Mathematieally, this is represented as: 

BUDGET.^, = (1 + r) • BUDGET. (2.5) 

where r > 0 is the random rate inerease eaeh year. Within the eode, the randomized range 
of r is [0, 0.1]. BUDGET= $8,000. This budget expires at the end of eaeh year similar 

to the DoD PPBS. There are four ways to alloeate budget: (1) buy intelligenee, (2) buy 
eounter-intelligenee, (3) aequire available weapon systems and (4) operate available 
inventory of weapon system. 

g. Intelligence 

Intelligenee is divided into offensive and defensive intelligenee. Offensive 
intelligenee purehased in year j allows knowledge of the eomputer’s offensive weapon 


19 



systems adjusted utils of year j (displayed prior to year j+1 eommitment). 

/' n \ n \ 

Y, COMP OB. 


Mathematieally, these values are shown as; YCOMPOA. 

Jj 


and 


V i=l 


Similarly, defensive intelligenee purehased allows knowledge of 

f « \ f n \ 

YCOMPDA. and ZCOMPDB. . 


V t = l 


v/=i 


h. Counterintelligence 

Counterintelligenee ensures that if the eomputer buys intelligenee, the 
information provided to them about the human player is less preeise. 


i. Acquisition 

A team may aequire additional units of any system already in the 
inventory at any time. Units of any new system may be aequired when foree information 
sheet indieates they are available and in any turn thereafter. In one or two (exeeptional) 
oases you may get an initial inventory free of oharge when the system first beoomes 
available. The maximum annual aoquisition rate for any partioular system is stated. 


j. Operations 

A team may operate any or all foroes in inventory at the start of a year. 
Operation is not mandatory for any of the systems. One oannot “mothball” units. Existing 
foree units not operated in any one year are lost from inventory. Units aequired in one 
year oannot be operated until the following year. 

2. User Interface Evolution 


Figure 2 displays the original TEMPO user interfaoe. The original TEMPO user 
interfaoe was a oommand oonsole where eaoh allooation was required to be submitted one 
at a time. This method did not aooommodate a ohange of mind—onoe an allooation was 
entered on a weapon type, it was unohangeable. This makes it diffioult to properly plan 
the budget. 
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Figure 3 through Figure 7 display and describe the modified TEMPO user 
interface. This version relied on Active Template Library (ATE) code. The code 
associated with this version developed a grid Microsoft Component Object Model 
(COM) Object as the interface, which was loaded within Microsoft’s Internet Explorer™. 
This modified TEMPO version allows the user to make decisions, determine impacts on 
cost and utils, and change things prior to committing the budget. This is a major 
enhancement to implementing a realistic application of the resource allocation process; 
however, this TEMPO version still retains many limitations along with the original 
TEMPO command console interface for successful SEVER implementation. These 
limitations are discussed further in the next chapter. 
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Figure 3. Updated TEMPO Graphical User Interface 
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Figure 4. Breakout of Existing Environment Information Display on the Updated TEMPO GUI 
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Weapon Availability/ Capabilities 
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Figure 5. 


Breakout of Existing Weapon System Information on the Updated TEMPO GUI 
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Figure 6. Breakout of Decision Execution Interfaees on the updated TEMPO GUI 
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Figure 7. Breakout of Decision Evaluation Information on the Updated TEMPO GUI 


B. SYSTEMS ENGINEERING VALUE EQUATION WITH RISK (SEVER) 

SEVER was derived and first implemented by Eangford (2006) under the process 
of ‘Rapid Systems Engineering,’ Now termed Value Systems Engineering. Value 
Systems Engineering is a scenario-driven approach that attempts to reduce the degree of 
uncertainty in predicting business success by structuring and analyzing the interplay 
between alternative business models, competitive strategies, and their resultant product 
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alternatives. It is a bottom-up, systematie, and highly iterative approaeh relating 
eompetitive strategies to alternative business requirements and eonditions for stakeholder 
sueeess (Langford, 2006). Langford (2006) utilizes SEVER as the fundamental tool to 
quantitatively prediet sueeess of partieular businesses. SEVER is closely related to Value 
Analysis and Engineering (Miles 1972). Both view a system as a collection of functions 
arranged in a hierarchical fashion. Details on Value Systems Engineering are located in 
Appendix A. 
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III. DEVELOPMENT METHODS 


In order to address the fundamental research question in this thesis, both a 
thorough understanding and detailed decomposition of the TEMPO model and the 
SEVER algorithm were required. Specifically, once this decomposition process began, 
the challenge undertaken was how to properly represent TEMPO output variables as 
SEVER input variables. Once achieved, more detailed questions surfaced. Answers to 
some of these questions were provided through actual software development. These 
questions include: 

What is required? 

• Represent performance, investment, function, and quality, as defined by 
SEVER, from TEMPO variables. 

• Determine weapon availability patterns. 

• Understand pWar and the value it provides. Is this realistic? 

• Characterize the computer’s weapon allocation decisions. Are these 
decisions consistent for each year from one game to the next? 

• Identify TEMPO software modifications required to incorporate the 
SEVER outputs. 

How can this be achieved? 

• Conduct many trials of the game. 

• Capture each trial’s output permanently. 

• Develop particular strategies to play against the computer. 

• Post-process trial data to determine the effectiveness of certain strategies 
(used to compare against the predicted SEVER value for each strategy, 
once implemented). 

Where possible, the answers to these questions are provided in this chapter. Other 
questions that could not be answered provide the framework for future work. 
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This thesis describes the SEVER algorithm, yet no formal execution of the 
algorithm or related software execution is presented. The work presented is only for a 
software package that will be used in the future to evaluate SEVER. 

A. APPROACH 

SEVER in its entirety is reformulated for each function and represented as; 

Risk _Vulnerability Performance* Quality (3 1) 

Threat Investment 

All terms are consistent with the definitions previously given. Regarding the units 
of each variable, performance is expressed in the same units as Hthreat (i.e., for TEMPO, 
‘Utils’). Quality is expressed as the same units of investment (i.e., for TEMPO, ‘$’). 

Risk 

Vulnerability is a probability without units. This defines the expression-in units of 

Threat 

performance (i.e., for TEMPO: ‘Utils’). In plain English, this representation states the 
ratio of the risk to all threats that would prevent the performing a particular system 
function is equivalent to (all else remaining constant); 

1. How vulnerable is the function to the composite threat —if the function is 
invulnerable to the threat, the risk of any threat to eliminate it is negligible 

2. How well the function performs —if the function does not perform well, the 
risk of any threat to eliminate it is negligible 

3. How much of a value is the function to society —if the function’s quality 
(i.e., value to society) is high, the risk of any threat to eliminate it is high 

4. How inexpensive was the function to develop —if the function was cheap to 
develop and implement, the risk of any threat to eliminate it is low since it 
can; 

a. be developed or implemented again 

b. provided redundantly 

In order to calculate the total system value, each function’s value must be 
quantitatively derived. Thus, applying SEVER to a system involves a top-down system 
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decomposition (collected using the functional decomposition method) and a bottom-up 
calculation of each function’s value. The final product is a summation of all the system’s 
function values. 

According to Langford (2006), combined with the investment made by a 
stakeholder, functions, performance, and quality requirements determine the operational 
capability of the product or service. Thus, considering a product or service as a system, 
the framework for calculating a system’s value is represented as the system’s operational 
capability given a certain required investment. Multiple systems attempting to achieve the 
same end goal can be compared using this framework. The system’s operational 
capability, or worth, is based on its functional view and is quantitatively measured in the 
same units as performance. 

Further, this framework can be extended to evaluate strategic decisions, or more 
appropriate for this thesis, a strategy. If a strategy is thought of as a system (i.e., the 
output of a particular strategy provides more value to society than others—society in this 
case is the player), the value of one strategy against another can be compared analogously 
to products and services using SEVER. 

1. Decomposed for TEMPO Application 

The following decomposition explains the how the calculation of consequence 
within the context of TEMPO is captured. By more formally representing investment 
algebraically as: 



where c is the incremental unit cost, tunit is the smallest applicable increment measure of 
time, and Tjotai is a total aggregate measure of time for a function, value per function can 
now be expressed as: 
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(3.3) 


In TEMPO, =Tj.^tai ■ ^unit represents one year and Tjotai represents the total 

/=! 

duration of the game. Units for 1 are Rearranging terms and multiplying by Pjotai / 
Piotai yields; 



P incremental represents the performanee for a funetion per year, where Protai 
represents the total performanee over the eourse of the game. Similarly, Qrotai is 
measured as the total quality of the funetion over the eourse of the game. This is the 
representation eurrently eonsidered for use with TEMPO. It eonsists of a produet of three 
major terms to align with the partieular outputs expeeted from TEMPO. V/E is ealeulated 
in the same units of P, Utils. 

Q is a quality funetion measured as a loss to soeiety (Taguehi, 1990) represented 
by (o.lO) below. The units of Q are the same as I; 

e(P) = /?-L.(P) (3.5) 

where (P) represents the loss funetion. Typieally, quality as a funetion of performanee 

is estimated using a quadratie loss funetion. A deviation from the optimal performanee 
results in a loss equivalent to that deviation squared. See Eigure 8 for the relationship 
between quality and performanee. Qualitatively, P is the maximum theoretieal measure 
of quality eorresponding to the optimal performanee level. This value represents both the 
developer and the user. 
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a. 


Derivation of Taguchi Loss Function 


Consider a representation of quality as shown in (3.6); 

Q^ + L,{Y) = B (3.6) 

where B is the maximum quality, Qm is the measured quality, and LfY) is the Taguehi 
Loss funetion, as stated in (3.7), 

L^{Y) = k{Y-my (3.7) 


where k is the loss faetor (derived empirieally), Y is the individually measured 
performanee, and m is the ideal target performanee. k is further deeomposed as; 


k = 


A 

¥ 


(3.8) 


where A is the average eost of eaeh unit returned/reworked and (5’is the range of 
variability in performanee; 

S = {UL-LL)/2 (3.9) 


UL and LL are the upper and lower limits away from the ideal target performanee, m, 
expeeted. 
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Figure 8. Generic Representation of Quality as a Function of Performance 


Thus, quality is represented completely as shown in (3.10); 

A 


Qt-P- 


( UL-LL 

I 2 


(y-m) 


(3.10) 


b. Commentary 

From this point, properly deriving the quality of each decision in the 
TEMPO game requires empirical effort based on statistical data. TEMPO Version 3 is 
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now capable of performing the groundwork analysis to derive the quality of eaeh 
deeision. This empirieal analysis to further define quality of eaeh deeision remains a 
future effort. 

TEMPO Version 3 involved the following development; 

• Creating an automatie gameplay feature that allows a defined 
number of games to be played with partieular strategies. 

• Developing the rule-based resouree alloeation strategies 

• Updating the GUI to inelude SEVER outputs 

• Condueting many games to prove the output of pWar is aeeurate 

These new eapabilities enabled TEMPO to automatieally eonduet a user- 
defined number of trials for SEVER to be statistieally evaluated. The variables required 
for SEVER ean now be empirieally derived and ealibrated to prove that the value 
determined by SEVER is aeeurate for a partieular resouree alloeation strategy employed. 
The TEMPO Version 3 software development effort is deseribed in greater detail in the 
next ehapter. 

B. EVALUATION 

Based on this researeh, the method ineorporated to perform preliminary testing 
ineluded eoding rule-based strategies to eapture a more detailed understanding of 
TEMPO and its potential applieation to SEVER. The ultimate goal is to allow SEVER to 
seore eaeh resouree alloeation deeision ‘on-the-fiy’ prior to the player eommitting his/her 
deeisions. This ean only be aeeomplished onee the value and risks for a given deeision 
are understood and quantified. 

Six different rule-based strategy types were developed. Of those, a total of 10 
individual strategies were exeeuted. Tum-by-turn data was eolleeted for eaeh exeeution 
for a total of 1000 trials. Eaeh strategy was stored as a separate MATEAB workspaee 
file. A MATEAB funetion, “eolleetStats.m” eolleeted, organized, and stored these 
separate workspaee files. The MATEAB funetion, “eompOutput.m” eomputed the output 
graphs for eaeh strategy used to eompare the results for eaeh strategy. The philosophy in 
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developing this eode, along with the previously deseribed MATLAB software, is to 
design in the flexibility to implement, run, and evaluate new strategies as they are 
derived. 

A box plot is used to present a high-level eomparison between eaeh strategy. The 
MATLAB funetion, ‘boxplot.m’ (part of the Statisties Toolbox) was used within the 
‘eompOutput.m’ funetion to generate Figure 29, page 74. This plot displays eaeh 
strategy’s “player net utils when war oeeurs” over 1000 trails. Values above 0 are player 
win s . Values below 0 are eomputer wins. From this plot, a statistieal eomparison of eaeh 
strategy’s performanee is presented. The data is represented in a box and whisker format. 
The box represents the 25th to 75th pereentiles; the line within the box represents the 
median; the whiskers represent the rest of the data not eonsidered outliers; and the 
individual plotted points (+’s) represent outliers, as determined by the MATLAB 
funetion. 
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IV. DEVELOPMENT, RESULTS, AND RECOMMENDATIONS 


This chapter describes the development of the software. A functional description 
of the new software package is provided. This description is separated into six major 
sections; an overview of the software paekage, the software integration challenges, 
TEMPO Version 3 development from TEMPO Version 2, MATEAB strategy 
development, the MATEAB analysis development, and the total integration. New 
features were eonsidered and implemented based upon the inputs required by the SEVER 
algorithm, the existing eapabilities of TEMPO, and the measurable outputs required by 
MATEAB to run the rule-based strategies and analyze all strategies. 

A. OVERVIEW OF THE SOFTWARE PACKAGE 

The new software eonsists of code written in both TEMPO.NET and MATEAB. 
This software package was designed to exeeute particular strategies within TEMPO.NET 
and eapture/present the results for a defined number of trials. Exeeuting a strategy 
eonsists of the user seleeting a desired strategy and number of trials to exeeute. Capturing 
the resulting output data required a TEMPO.ATE model update and MATEAB software 
development. 

There were two major features required for TEMPO to gather the appropriate 
measures to implement SEVER. The first feature was for TEMPO to automatieally write 
an output log file at the end of eaeh game so this data could be permanently stored. The 
seeond feature of TEMPO.NET is the ability to automatieally play a defined number of 
games. 

The first feature already existed in the existing TEMPO ATE version. A log file 
for eaeh game was provided in an extensible Markup Eanguage (XME) format. 
Examples of this log file for a typieal game’s first year is presented in Eigure 9 and for a 
typical game’s last year is presented in Eigure 11. The highlighted fields are those that 
required modification. In order to store the appropriate data for analysis, some 
modifications to the data output in this file were required. All weapon inventories and 
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maximum numbers to purchase were incorrectly listed. For example, Year n+1 data was 
listed where Year n data should be. This is believed to be a bug in the existing ATL 
version and was updated by: (1) storing the previous year’s weapon data, (2) executing 
that year’s turn, and (3) writing the stored ‘inventory/maximum acquisition’ (or, weapon 
availability) data to the log file (along with the current year’s allocations) after 
committing to those weapon allocation decisions. Without incorporating this fix, 
comparison between weapon availability data and the allocation decision for every year 
would not be possible. The data associated with the first year’s weapon availability is lost 
and the data associated with the year that war occurred is unnecessary as it reflects 
weapon availabilities after war already occurred. 

The second update was implemented to facilitate data analysis, but it was not 
mandatory, ft consisted of displaying the budget left after the turn was executed, rather 
than the original budget for the start of the turn. This was convenient for the MATLAB 
analysis tool, as the actual budget left is considered important rather than the starting 
budget. In either case, the starting budget could be calculated assuming the weapons 
operated and acquired for that turn were accurate (as accomplished by the first update) or 
vice versa. Associated samples of the new log file are shown in Figure 10 and Figure 12, 
for a typical game’s first and last year, respectively. The modified fields are highlighted 
for clarity. 
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<VOiMi mmrn^ir A««la*r 0 o^’or Typpa*0‘ SubCypt-'O' tnyonca*20* MacAcqa'lS* ibqC«ita*7ft* OpCo««*lM‘ UtSli«M20’ 0plidA*0* YMrAv«Mbli«*0* /> 

<y9ktm Nltma*!* Ay^a'O* 0.0«'0* Typ^a*!* Subtypta'O* ln«Ma*40* M4«4cqa*25* 4cqCCtla*50* 0pCO4la*10* Ubi»«*20* OpMa'O* •OMiMa*0’ ypfA aiUbHa l* /> 

<ykim Nlim«*2* Av^a't* o 0«*1* Ty^'O* Subtfp«a*0* tnv«A(aMOO* NlKA^a*25' AcqC44ta'40' OpCOtCa*20' UtH|a*tS' Op«Ma*0* OO^Ma'O* PMr4«tt44bl4«*0* f> 

<VPI|I44 lli.ma*3‘ A^m'or O 0«*1* Typ*.'!* Subtypta'O* tfiv«n(a*20* Mac4<q-'25* AcqCMiaMOO' 0pC44ta*M* UtMa'SO* 0pM>'0* totgbta'O* Ytiri fbtblfM* /> 

<Vbl«Mi NitfIMU' Av44a*0* O 0«'0* Typ^a'O* Subtypta'l* VwnXm^Or H«KACqa*3S* AcqC04Ka-7V OpCptta'SS* Mllta'40* Optflda'O* •Otffbta'O* ypf Ai HbHa^a* f> 

<Vbtw4i Ikma'S* A»44a'0* 0 0.*0* Typ»a*l* SubtyptaT ln^.*0* HtxkqaMS' AcqC4ft-*2M* 0pCo4la»2S0* UbbaNOO’ Optod-’O* •6w9ht.*0* V4vA««iUbl»a*r /> 

<^iy«4 bufiia*** A»44a*0*0.0«*l* Ty^'O* SuMypta*!* tnypnCa^O* NtxAcqa*2S' AcqCotta’TO* OpCOfKa*SO* Mftta*t2S' 0ptida*0* •owgbt-'O* IP^vbibbba’a* f> 

<yMl lbma*r Avida'O* 0 Ot*r Typo«*r OuKypta*!* tnvonCa'O* NiUcqa'lO' AcqCOfta'lOO' 0pCo«a*100’ U0lla*)00‘ OpMa'O* OOMBma'Q* /> 

<VMll« N«ima*r A«44a*0* 0 0»‘0r Typpa'o* Subtypaa-a* lnv«4a*(r HixAcqa'lS* AcqCMta'M* OpCoataUO* UWta’lOO* Oplida*0' towpqa'o* >P^^4<ibHa-y /> 

<VPlU44 IKma-p* A«44a‘0*0 O^'O* Typ»a*t* Subtypfa*!* IflaMCa-y M«KAcqa*lS‘ AcqCO4ta*lP0* OpCMta’lU* UOHa'SOO* 0p4»da*0* bew 0 Ma‘O* yp4rAni4lbi|a*S‘ /> 

<VblM«i lli.ma*io* A««tta*0* O Da*l* Typ4a*0' SwbtyMa*2* lnv«ma*0* MaU4qa*2S* AeqCcaiaMOO' OpC04la’50* UOba'aOO* OpMa'O* iewQbta'O* YMTA i4l6i|a*a* /> 

<VPlM44 Nlima*ll* A««lla*0* 0 Oa*l* Typ4a*t* Sobty94a*a* lnv«nCa*0* M4«>^a*2S* ACpCCtta']25* 0pC0fKa‘50* UtMla*lSO* 0p*atfa*0* Oew0bta*0* PMrA««44bl»a*4* /> 

<AMap0M> 

</Tpmpoypbr> 

• < T 4iwppypar ty4» *CbmpMtar* npmpa'Rofi* <Mb»a*SM« 19 Opc 2000 14:04:^5 -0500* pnMrpnmpnt f(4a*Mivlrofim«nt^.txt*> 

« < T 4> i ipoP>ar t^4i •biwnfi* fi4m4a*4o«i* 4a4aa*S»t, 13 0«c 2004 14:04:25 -OSOO* anvlroomant IM4a**n<4r(Mim«ff»t_4.txt‘> 

• <T4flipo>lMr tippa*compvt«r* n4m«a*Rofi* 4Atia*S«t. 19 0«c 2004 14:04:25 *0500* inarowma w l ftaa*n>4ro f im n t,A.!Mt*> 

« <TliHipoyp4i t)p4i 'bumti* n«m 4 a*lon* dabaa'iat, 13 0«c 2004 14:04:25 -0500* anvironmant M4**«nwlrofimMit_i.txt'> 

« <T4fwpcPMr tip4a‘computer* namoa'Ron* d4tea*Sot, 19 Ooc 2004 14:04:25 *0500* •nvtronmant ftoa*on^rofimMt_A.tJrt*> 

• <T4fnpo>44r tvoa'humon' nontea'ton* dAtpa'Sot. 13 Ooc 2004 16;04;23 *0500* pnvlronmpnt Mir*4nt4ronm4nt_B.txt’> 

</Tampc> 
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<?xml verelons"!.©' encoding s’UTF*8“ ?> 

- <Tempo> 

- <TempoYaar types'camputer" names'* dates'Sun, 21 Dec 2008 09:59:00 *0500' environment nies*'conflg\envlronment_A-txt*> 

- <Env1ronmGnt> 

<Header Hl="Year‘' H2="Pwar“ H3=“Budgef H4=“NGtUtil9“ H5="ICost[0]" H6="ICost[D]" H7=“CICoBf H8="IBought[0)' H9="IBought[D]" H10="CIBought" /> 

<values Years'O" Pwars-o.l" BudgeC="1025" Netutlls="0" ICostO="100" ICostDs-lOO" CICost=“30Q“ IBoughtO="0“ IBoughtD="0" CIBought='0" /> 

</Envlronment> 

- <Weapons> 

<Haader His"#* H2s*Avair H3s*0/D" H4s*Type" HSs-Subtype" HBs-Invent" H7s*MaxAcq" HSs-AcqCost* M9s-OpCo5t' HlOs'Utils' Hlls-Opted" H12s-Bought" H13s*YearAvailable‘ /> 
<Values Nums'O* Avaiis'True" 0.0s"0‘ Types*0* Subtype=’0* Invents*20' MaxAcqs'lS* AcqCosts'75'' OpCosts'lSO* Utllss*120' 0ptiads''20‘ 8oughts*13" YearAvallables'O* /> 

<Values Num="i” Avalla'False’ 0_Ds*0' Types"l” Subtypes'O" lnvent=*0" MaxAcqs*25* AcqCost="50" OpCosts'BO" Utils=*20’ Opteds'O" Bought=*0" YaarAvaliables'l* /> 

<Values Nums''2' Aval]s''True' 0_Ds''l' Types'O* Subtype=*0' Invents'lOO’ MaxAcqs''25’ AcqCosts'AO* OpCosts*20* Utllss"15" Opteds'lOO” 8oughts''25” YearAvallables'O" !> 

<Values Nums’3" Avaiis’False* 0_Ds"l" Typea"l" Subtypes'O" lnvent=“0‘‘ MaxAcqs*25* AcqCosts''lOO* 0pCo3ts*60" uaiss"50“ 0pted="0‘ Boughts'O* YearAvalleblea"!" /> 

<Values Nums"#" AvailB"Fal9e‘ 0_Ds"0“ Types"©" Subtype«"l" Invents'O" NaxAcqs“35" AcqCosta'TS" OpCosts“35* Uttlss"60" Opteds’O" Boughta'O" YearAvallables"2" /> 

<values Num="S" Avaiis'False" O D=*0" Type="l" Subtype="l" lnvent="0* MaxAcqs'is" AcqCost="200‘ 0pCost="2S0” utHss’eoo" opteds-o* 8ought="0* YOBrAvailable="2‘ /> 

^Values Num="6" AvaH="Fal9e“ 0_D="1" Type="0’ Subtype="l" lnvent=‘0" MaxAcq=’25" AcqCo8t="70" OpCc»8t=’50’ Util8="125" 0pted="0" Bought=:"0" YearAvallable="2" /> 

<Values Num="7‘ Avail="Fal9e' 0_D="1' Type=:"l" Subtype=*l'' lnvent="0" MaxAcq=“15" AcgCosts^lOO" 0pCost=‘100* Utllss’SOO" 0pt»d="0* Bought="0' YearAvallable="4* /> 

<Values Num="8" Avall="False‘ O D='0“ Type="0“ Subtype="2" lnvent="0'' MaxAcq="15" AcqCost="®0‘' 0pCost="40“ Utlls="l00‘' 0pted="0“ Bought="0“ YearAvailable=*3' /> 

<Values Num="9* AvaH="False' 0 D='0" Type="l" Subtype="2" lnvent="0" MaxAcq="15" AcqCost="300* OpCost=*125’‘ Utils="300" 0pted=*0" Bought=“Q" Ye8rAvailab!e="5‘ /> 

<Values Nums'iO" Avalls'False" O.Ds'l* Types’O" Subtypes"2" Invents’O' MaxAcqs’25" AcqCoets'ioO" OpCosts’BO" Utilss’200'' Opteds'O* Boughts’O* YearAvailables’2' /> 

<Values Num="ll" AvalU'False" 0_D="1" Type=*l" Subtype="2" lnvent=“0" MaxAcqs"25" AcqCo8tB'125" 0pCt)3t="50" Utils="150" 0pt)ed="0‘ Bought="0" YearAvailable="4" /> 

</Weapon5> 

</TempoYGar> 

> <TempoYear types'human" names’auto" dates’Sen, 21 Dec 2008 09:59:00 -0500" environment flles"conflg\envlronment_B.txt"> 

- <Envlronment> 

<Header Hls’Year" H2B"Pwar" H3B"Budget‘‘ M4B*NetUtil3" HSB"ICost[0]" H6B"ICost[D]* H7B‘ClCo5r H8B"IBought[0]‘ H9s"IBought[D]" H10s"CIBought" /> 

<values Years'o" pwars’o.i* BudgeCs’675* Netutilss’o’ icostOs’iDO” icostD=*ioo" cicosts*300" iBoughtOs’ioo' iBoughtDs’ioo” aBoughts'o’ /> 

</Envin5nment> 

• <Weapons> 

<Header His"#' HZs'Avail* H3s-0/D" H4s"Type" HSs’Subtype" HBs’lnvent" H7s*MaxAcq" Hes'AcqCost* H9s*OpCo5t" HlOs 'Utils" Hlls’Opted" H12s-Bought" HUs'YearAvailable* /> 
<Values Nums'O" Avalis"True" 0 Ds"0" Types'O" Subtypes'O" Invents'20" MaxAcqs'15' AcqCosts*75" OpCosts'isO" Utiiss*i20" 0pteds"20' Boughts*l5" YearAvailables'O" /> 

<Vaiues Nums'i* Avalls’False* 0_Da*0" Types’i* Subtypes'O" invonts’O" MaxAcqs'25" AcqCosts"50" 0pCosts"30“ Utllss’20* Opteds’O" Bought*’©" YearAvallables'i" /> 

<Vatues Nums"2' AvalU'True" O.Ds’l" Types’O" Subtypes’O" Invents‘lOO* MaxAcqs"25* AcqCosts*40' 0pCosts*20‘ Utliss*l5" Opteds’lOO" Boughts’25" YearAvailables'O" /> 

<Vaiue5 Nums"3' Avaiis’False" O.Ds'l" Types'l* Subtypes'O" Invents"©" MaxAcqs*25" AcqCosts'loO" OpCosts‘60" Utilss’50" Opteds’O" Boughts’O' YearAvallables'i' /> 

<Values Nums"4’ Avalls’False' 0 Ds’O" Types"©" Subtype='l" lnvent="0" MaxAcqs*35" AcqCosts’75" 0pCosts"35" Utllss’60' Opteds’O" Boughts’O" Ye8rAvallable3’2' /> 

<Values Nums’5’ Avalls’False' 0_Ds’0’ Types’l' Subtypes’l’ Invents’O" MaxAcqs'lS" AcqCo5ts’200' 0pCo5ts’250" Utllss’400'’ Optseds’O’ Boughts'O" YearAvallables’2’ /> 

<Values Num="6* AvaiU'False” 0_D='l" Type="0’ Subtypes'l" lnvent="0'' MaxAcq=*25" AcqCo8t="70" OpCosts'SC" Utils="125'' 0pted="0* Bought="0* YedrAvaildble="2* /> 

<Values Num="7* AvaiU'False* 0_D=’l' Type="l" Subtype='l" lnvent="0" MaxAcq="15" AcqCost=''100* 0pCost='100" Utils="300" 0pted=*0* Bought='0* YearAvailable="4' /> 

<Value5 Nums’S' Avaiis’False' 0 Ds’O" Types’O" Subtypes"2" Invents’O" MaxAcqs'15" AcqCosts’60" OpCosts’40" Utllss’lOO" Opteds’O' Boughts’O' YearAvallables*3* /> 

<Values NumB’9' Avalls’False' 0 D*’©' Types’l* Subtypes'2" Invent*’©" MaxAcqs'15’ AcqCosts’SOO* 0pCosts'i25” Utllss’300'' Opteds’O' Bought*’©' YearAvailables’5' /> 

<Values Nums’iO" Avalls’False’ 0 Ds'l' Type*”©’ Subtypes’2’ Invents”©* MaxAcqs’25" AcqCosts’ioO” OpCosts’SO" UtllS8’200" Opteds’O' Boughts’O* YearAvallables”2" /> 

<Value5 Nums’ll’ AvaiU'False’ 0_D='l' Type»'l“ Subtype=“2" Invents"©" MaxAcq»"25" AcqCoBt='125' 0pO)5t="50" Utilss'lSO" Opteds'O" Bought='0" YeBrAvailable="4' /> 

</Weapons> 

</TempoYear> 

•f <TempoYear types'computer’ names" dates'Sun, 21 Dec 2008 09:59:00 -0500" envlronment_nies'config\envlronnient_A.txt"> 

•f <TempoYear types’human* names'auto" dates’Sun, 21 Dec 2008 09:59:00 -0500" environment_flles*conflg\envlronment_B.txt”> 

•f <TempoYear types'computer" names" dates’Sun, 21 Dec 2008 09:59:00 -0500" environment_nies*config\environment^.txt"> 

•f <TempoYear types'human" names'auto" dates’Sun, 21 Dec 2008 09:59:00 -0500" envirDnment_files'config\environment.B.txt'> 

•*> <TempoYear type='computer” names" date=’Sun, 21 Dec 2008 09:59:00 -0500" environment flles’conf1g\envlronment_A.txt*> 

•*> <TempoYear type='human’ name="auto' date="Sun, 21 Dec 2008 09:59:00 -0500" environn)ent_file=*config\environment_B.txf> 

^ <TempoYear type="corYiputer’ name=" date="Sun, 21 Dec 2008 09:59:00 -0500" envlronment_nie="config\envlronment_A.txt’> 

•f <TempoYear types'human" names'auto' dates’Sun, 21 Dec 2008 09:59:00 'OSOO" envln3nment_flles'config\environment_B.txt'> 

</Tempo> 
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<?xml versionr'1.0" encoding =*UTF-8’ ?> 

- <Tempo> 

+ <TempoYear types "computer" name»'‘Ron“ date&'Sat^ 13 Dec 2008 16:04:25 -0500“ envlronment_files"environment_A.txt“> 

* <TempoYear type="human" name="Ron" date-'Sat 13 Dec 2008 16:04:25 -0500" environment flles"environment_S.txt"> 

* <TempoYear types"computer" names"Ron" dates'sat 13 Dec 2008 16:04:25 *0500“ environment_files"envfronment_A.txt'> 

* <TempoYear type=:"human" name="Ron“ date=“Sat, 13 Dec 2008 16:04:25 -0500“ envlronment_file=“environment_B.txt"> 

* <TempoYear types’computer" names"Ron" date='Sat, 13 Dec 2008 16:04:25 -0500“ environment_flles"envlronment_A.txt"> 

* <TempoYear types'human" name="Ron" dates'Sat, 13 Dec 2008 16:04:25 -0500“ envlronment_filea“envlronment_B.txt“> 

- <TempoYear type=“ computer" name=“Ron” date=“Sat^ 13 Dec 2008 16:04:25 -0500“ envlronment_file=“env{roffiment_A.txt‘> 

- <Envlronment> 

<HeaderHls“Year" H2s-Pwar‘ H3 = "Budget“ H4s"NetUtlls“ H5s“lCost[0]" H6s"iCost[D]" H7s"CICost“ H8="IBou9ht[0]“ H9s-iBought[D]“ H10s"CIBought‘ /> 

<Values Years’S" Pwars"0.165201‘ Budoets’9398" NetUtilss“1300" ICostOs“100" ICostDs’lOO' aCosts“300* I8ought0s'0“ IBoughtDs’O' CI6oughts"0" /> 

</Environment> 

« <Wdapons> 

<Header HI-"#" H2-*Avair H3="0/D" H4="Tvpe' H5-"Subtype" H6="Invenr H7="MaxAcq" H8="AcqCost" H9="OpCost" HiO="Utlls“ Hll-"Opted" H12-"Bouglir H13-"YearAvailable' /> 
<Values Numa"0" Avails"!" 0_Da"0" Type="0“ Subtypes"0" Invent3"20" MaxAcqs"l5’ AcqCosts"75" 0pCosts"l50“ Utllss"l20“ 0pted = "0" Boughts’O" YearAvailables’O" /> 

<Values Num-'l" Avail-"1" O 0-"0" Type-'l* Subtype-'O" Invents"40" MaxAcq»'25" AcqCost-"50" 0pCosts*30" Utllss"20" Opted-"65* Bought-"!?" YearAvailable-"!* /> 

<Values Num="2" Avail-"!" 0_D="1" Type=”0" Subtype-"0" Invent="100" MaxAcqs"2S" AcqC©st="40" OpCost="20’ Utlls="15" 0pted»*4S" Bought-'O" YearAvallable="0" /> 

<Values Num-'3" Avail-"!’ 0 D-"l" Type-'l* Subtype-"0" Inventa"20" MaxAcqs”25" AcqCost-"100" OpCost-"60" Utlls-"50" Opted-’IS" Bought-*0" YearAvailable-'l" /> 

<Values Num=“4" Avall="l" 0 D="0" Type-“0" Subtype="l" lnvent="0" MaxAcq="35" AcqCost="75“ OpCost*”35" Utl!s="60’ 0pted-"0" Bought*'©" YearAvallable*'2" /> 

<Values Nums’5" Avails'i" o.D-'o" Type-’i’ subtype-’i" invents"0“ MaxAcq-'is" Acqcosts"200" OpCosts"250“ U0lss"400‘ Opted-'O" Bought-*l4" YearAvallables"2" /> 

<Vaiues Nums“6" Avail*"!" O D*"!" Type*'0" Subtype*"!" Invent*"0“ MaxAcq*"2S“ AcqCost*"70’ OpCost*'50" Utlls*"125" 0ptad*"0* Boughts'lS" YearAvailable*"2’ /> 

<Values Num="7" Avall»"0" 0_D="1" Type="l" Subtype-"!* Invent="0" MaxAcq="15" AcqCost="100'‘ OpCost="100“ Utlls="300" 0pted = "0" Bought- "0" YearAvallable-"4" /> 

<Va(ues Num="8'' Avail="l’ 0 D="0" Type="0" Subtype="2* lnvent="0" MaxAcq=*15" AcqCost=*60" OpCost="40" Utils=*100" 0pted = "0* Bou9ht="0'' YearAvailable="3" /> 

<Vaiues Num*'9" Avail*"0" O D*"0" Type*"l" Subtype*"2" lnvent*'0' MaxAcq*"15" AcqCost*"300" OpCost*"125' Utlls*"300“ Opteds'O" Boughts’O" YearAvallable*"5" /> 

<Values Num="10“ Avall="l" 0_D="1“ Type="0" Subtype="2“ lnvont="0" MaxAcq="25" AcqCost="100" 0pCost="50“ Utils="200' 0pted = "0" Bought=‘9“ YearAvailable="2" /> 

<Values Num="ll’ Avail="0" 0 D="l" Type="l" Subtype="2" lnvent="0" MaxAcq="25" AcqCost=’125" OpCost="50" Utll8="150" 0pted = "0" Boughts’O" YearAvailabla="4" /> 

</W©apons> 

</TempoYear> 

- <TempoYear type*"human" name*"Ron" date*"Sat, 13 Dec 2008 16:04:25 -0500" environment files"environment_B.txt"> 

- <Envlronment> 

<HeaderHl = "Year" H2="Pwar" H3="Budget" H4="NetUtils" HSs"ICost[0]" H6="ICoat[D]" H7="CICosr H8="IBought[0]" H9="IBought[D]" H10="CIBoughr /> 

<Values Year*"3" Pwars"0.157745" Budget*"8709" NetUtils*’-1300’ ICost0*"100" ICostD*"100’ acost*"300' IBought0*"0" IBoughtD*’0' aBought*"0" /> 

</Environ ment> 

- <Weapons> 

<Header HI*"#" H2s"Avall" H3s"0/D" H4s"Type" H5s"Subtype“ H6s"Invent" H7s"HaxAcq" H8s"AcqCo5t“ H9s"OpCo5t" HlOs-UtHs" Hlls’Opted" H12s"Bought" H13s"YearAvallable' /> 
<Values Numa"0" Avails"i" o_Ds“0" Type-"©" Subtype*"©" Invent=’20" MaxAcq="15‘ AcqCo5ts’75“ OpCost="150“ Ublss"120“ 0pted="0“ Bought*"©" YearAvailable*"0“ /> 

<Values Num="l" Avall="l" 0 D="0" Type="l" Subtype="0" Invent="40" MaxAcq=’2S" AcqCost="50" 0pCost=“30“ Utlls='20" Opted-"©" Bought-*©" YearAvallable-"!" /> 

<Values Num="2" Avail*"!’ 0_D»"1* Type*"©” Subtype="0" Invent*"100" MaxAcqa"2S" AcqCcsts"40" OpCo5ts"20" U0ls="15" Opted*"©" Bought*"©" YearAvallable* “0" /> 

<Values Num="3“ Avail*"!’ 0_D*"1’ Type*"!" Subtype*"©" Invent*’20“ MaxAcq*"25" AcqCosta’lOO" OpCo5ta"60“ Utllsa’SO" Opted*"©" Bought*"©" YearAvailable*"!" /> 

<Values Num*"4" Avail*"!’ 0 0*"0“ Type*"0" Subtype*"!" Invent*"©" MaxAcq*"35" AcqCost*"75' OpCost*"35" Utlls*’60“ 0pted*“0“ Bought*"©" YearAvallables"2" /> 

<Values Num="S" Avall»"l’ o.D="0" Type='l" Subtype-’l" lnvent=’0" MaxAcq»"l5" AcqCost="200“ OpCost=’250" UtJls="400“ Opted-*©* Bought-"©" YearAvallable-" 2" /> 

<Values Num="6" Avail="l* 0 0="1* Type**©" Subtypes’!" Invent=’0" MaxAcq=’2S’ AcqCo3t=’70* 0pCost=*50" Utlls=’125'' Opteds’O" Boughts’O" YearAvallables’2" /> 

<Values Num="7" Avalls'D” O 0*"!" Type*"!" Subtype*”!" Invent="0" MaxAcqs’15" AcqCosts’lOO" OpCosts’iOO" Utllss”300‘ Optsda’O* Bought*"©" YearAvallable*" 4“ /> 

<Values Num="8‘' Avall»"l" 0_D="0" Type="0" Subtype="2" lnvent=’0" MaxAcq-’lS" AcqCost="60" OpCost="40" Utlls-’lOO" 0pted-"0' Bought-*0" YearAvallable="3" /> 

<Values Nums’9" Avails’©" O Ds’O" Types’!" Subtypes’2* Invents’©" MaxAcqs’15* AcqCosts"300" OpCosts’l2S* Utllss’300' Opteds'O' Boughts’O" YearAvailables’5’ /> 

<Values Num*"10“ Avails'i" 0 Ds"l" Typea’O" Subtypa="2" Invent*'©' MaxAcqs'2S" AcqCost="100" 0pCost*'50" Utlfss'200' Opted*'©’ Bought*"©" YearAvallab!es“2’ /> 

<Values Numa’ll" Avail*"©" O.Ds'l" Typea’l" Subtype*"2" Invent*"©" MaxAcq*"25" AcqCosts’125" OpCosts'50" Utllsa'lSO" Opted*"©" Bought*"©" YearAvallable*’ 4" /> 

</Weapons> 

</TempoYear> 

</Tempo> 
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<?xml verelon»"1.0" encodings"UTF-8’ ?> 

- <T«mpo> 

* <TempoYear type=*cofnputer'' name='"' date='Sun, 21 Dec 2008 09:59:00 -0500* envlronment_fite=*conflg\environment_A.txt‘> 

<TempoYear t^e=*human’ name-''auto" date=*Sun, 21 Dec 2008 09:59:00 -0500" envlrDnrnent_flles‘conflg\environmen1:_B.txt‘> 

4- <TempoYear type=*computer" names*” date=’Sun, 21 Dec 2008 09:59:00 -0500* environment_file="conflg\envlronment_A-t>ct*> 

* <TempoYear types"human‘ names''auto" dates*Sun, 21 Dec 2008 09:59:00 -0500" envln3nment_files”conflg\environment_B.txt*> 

* <TempoYear types'computer" names** dates'Sun, 21 Dec 2008 09:59:00 -0500* environment f)les''conflg\envlroninent_A.txt*> 

•f <TempoYear types''human* names''auto" dates'Sijn, 21 Dec 2008 09:59:00 -0500" environment files”config\environment_B.txt”> 

<TempoYear types’computer” names** dates’Sun, 21 Dec 2008 09:59:00 -0500* environment files *conflg\environinent_A.txt*> 

4- <Ten>poYear type="human" names*auto" date=‘Sun, 21 Dec 2008 09:59:00 -0500" environment_flles*c(Hifig\environment_B.txt*> 

- <TempoYear types "computer* names** dates'Sun, 21 Dec 2008 09:59:00 -0500’ envlronment_nies*conflg\environment_A.txt*> 

- <Envtronment> 

<Header Hl«"Year" H2-"Pwar” H3«"Dudger H4«*NetUtlls" H5s"ICost[0]" H6s'IC05t[D]* H7s*ClCost* H8s"IBought[0]’ H9s"IBought[D]" HiOs’CIBought" /> 

<Values Yeors“4* Pwars-0.197475168633393" Bud9ets*274" NetUtll*s“-2930* ICostOs-lOO’ ICoetOs-lOO" CICo8ts-300" IBoughtOs-O’ IBoughtDs'O" QBoughts’O" /> 

</Envitt>n ment> 

• <Weapons> 

<Header Hi^"#* H2s"Avall‘ HS^'o/D" H4s*Type" HSs’subtype” H6s*invent" H7s"MaxAcq'' HSs’AcqcosT H9s"opcost" HiOs’utlls" Hiis’opted" Hi2s"8ouoht" Hi3s"YearAvallable‘ /> 

<Values Numa’O* Avalla’True" 0_Ds"0" Typea’O" Subtypes’O* lnventa“0" MaxAcqs*i5* AcqCosts“75" OpCosta"150" Utllss*i20" Opteda*0’ Boughta’O* YearAvallabtea’O* /> 

<Values Nums’l* Avalls'‘True' 0 Ds"0" Types'i* Subtypes'O” Invents”90' MaxAcqs"25* AcqCosts’50" OpCosts*30" Utllss*20’ Opteds’SS* Boughts*o* YearAvallables*i* /> 

<Values Nums"2' Avalls''True" 0 Ds"l* Types”0" Subtypes'O’ Invents'O" MaxAcqs”25* AcqCosts”40" 0pCosts”20* Utllss”l5’ Opteds'O" Boughts'O" YearAvallables'O" /> 

<Values Nums" 3 ' Avalla’True" 0 D*"!* Type**!" Subtypes'O" Invents*47’ MaxAcqs"25“ AcqCosts"i00" OpCosts'60* Utllsa*50" 0pted»’29" Boughts’O" YearAvallables"!" /> 

<Values Nums"4* Avails’True" 0_Ds"0" Types“0" Subtypes’l" Invents’O" MaxAcqB‘35" AcqCosts"75’ 0pCo3ts“35* Utllss"60’ 0pt3^*"0" Boughts’O" YearAvailables’2" /> 

<Values Nums"5’ Avalls’True" 0_Ds"0" Types’i" Subtypes’l" Invents"12“ MaxAcqs“l5" AcqCosts*200’ OpCosts“250" Utlls="400" Optods’6" Bought=’l* YearAvailables*2" /> 

<Values Num="6’ Avall = "True" 0 D="l" Type=’0" Subtype=*l' lnvent="0" MaxAcq**25" AcqCost="70" OpCosts’50" Utlls»*125" 0pted»’0* Bought»"0' YeorAvailable="2" /> 

<Values Num="7’ Avalls’True" 0_D="1* Types’i" Subtypes’l" lnvent="0” MaxAcqs’iS" AcqCosts’ioO" OpCosts'lOO" Utllss"300* Opteds’O" Boughts'iS" YearAvallable=’4" /> 

<Values Num="8’ Avalls’True" 0_D="0“ Types’O" Subtype=“2" Invents’O" MaxAcqs’i5“ AcqCosts*60“ 0pCosts’40" UtJIss’lOO" 0pted=’0’ Bought="0* YearAvallables "3" /> 

<Values Nums"9' Avails*False’ 0_0s*0' Types’i* Subtypes*2’ Invents’O" MaxAcqs*l5" AcqCosts"300’ OpCosts*i25" Utllss”300” Opteds’O" Boughts’O’ YearAvallables" 5’ /> 

<Values Nums’lO" Avalls'True’ O Ds-l’ Types’O* Subtypes’2’ Invents’O' MaxAcqs’25’ AcqCosts’lOO’ OpCosts-50‘ Utllss'200‘ Opteds'O" Boughts’O" YearAvailables-2’ /> 

<Values Nums’ll" Avalls'True’ 0_Ds*l* Types’i* Subtypes’2“ Invents’O’ MaxAcqs"25’ AcqCo5ts'l25’ OpCosts*50’ Utilss*l50’ Opteds’O" Boughts"l4* YearAvailables*4’ /> 

</Weapon5> 

</TempoYoar> 

- <TempoYear types’human" names’auto" dates'Sun, 21 Dec 2008 09:59:00 -0500" environment flles'conflg\envlronment_B.txt”> 

- <Envlronment> 

<Header Hls’Year" H2s’Pwar" H3s"Budget" H4=’NetUtils" H5s’ICost[0]" H6s’ICoBt[D]’ H7s‘ClCoet‘ H8=’IBought[0]' H9s’IBought[D]" HlOs’ClBought" /> 

<Values Years'4’ Pwars’0. 218926429879326’ 6udgets'24’ NetUtll8s’2930" ICostOs’lOO" ICostDs'lOO" aCosts’300’ IBoughtOs’lOO" IBoughtDs’lOO’ CIBoughts'O’ /> 

< /Environ ment> 

- <Weapons> 

<Header His*#" H2s’Avair H3s*0/D" H4s"Type’ H5=’Subtype" HBs'lnvent" H7=’MaxAcq’ H8s*AcqCost’ H9="OpCost" H10s"UtllB’ Hlls’Opted" H12s"Bought" H13s’YearAvailable’ /> 
<Values Nums’O" Avalls'True" 0_0s"0" Types’O" Subtypes’O" Invents'O" MaxAcq = "15" AcqCost=’75’ 0pCosts*l50* Utllss*l20’ Opteds*0' Boughts’O" YearAvallables’O" /> 

<Values Nurns*!’ Avalls’True" 0 Ds"0" Types’i" Subtypes’O" Invents’llS* MaxAcqs*25" AcqCosts’50' OpCosts*30" Utllss'20" 0pted=*115" Bought=*25" YearAvallables’ 1* /> 

<values Nums’2’ AvalU'True" 0_Ds"l’ Types’O" Subtypes’O" Invents’O" MaxAcqs’25" AcqCosts’40'' OpCosts*20" UDIss’lS" Opteds’O" Boughts’O" YearAvallables’O* /> 

<Value5 Nums‘3" Avalls'True" 0_Ds’l’ Types’i’ Subtypes'O" Invents’l' MaxAcqs’25" AcqCosts’lOO" OpCosts’BO" Udlss'SO" Opteds’l* Boughts’l' YearAvallables’l' /> 

<Values Nums*4" Avalls’True" 0 Ds"0" Types’O" Subtypes’l" Invents’O" MaxAcq = "35" AcqCost="75" OpCosts*35" Utlls="60" Opteds’O" Bought="0" YearAvallables" 2* /> 

<Values Nums'5" Avalls'True" 0 Ds'O" Types’i’ Subtypes’l' Invents’22' MaxAcqs’15' AcqCosts’200" 0pCosts*250’ Utllss'400* Optods'iS' Boughts’O" YearAvallable='2’ /> 

<Vatues Nums‘6’ Avalls'True" O Ds’i’ Types'O" Subtypes'i’ Invents'O" MaxAcqs'25" AcqCosts’70" OpCosts*50" Utllss’i25’ Opteds'O" Boughts'O' YearAvallables’ 2’ /> 

<Value5 Nums’7" Avalls'True" 0_Db"1" Types’i’ Subtypes’l" Invents’O" MaxAcqs’15" AcqCosts’lOO" OpCosts’lOO" Ublss’SOO" Opteds'O’ Boughts'O" YearAvallables’4' /> 

<Value5 Nums’8“ Avalls'True" 0 _Db" 0" Types'O" Subtypes’2" Invents'©" MaxAcqs’15’ AcqCPsts’BO" OpCo5ts’40" Utllss’ioO" Opteds'O" Boughts'O’ YearAvailables’3“ /> 

<Vaiues Nums'9" Avalls'False' O Ds'O' Types’i” Subtypes’2’ Invents'©" MaxAcqs’15’ AcqCosts’300’ OpCosts'i25' Utllss’300' Opteds'O" Boughts'O' YearAvailables'5' /> 

<Vaiues Nums’lO' Avalls'True' O Ds'l’ Types'O" Subtypes”2’ Invents’©’ MaxAcqs'25” AcqCosts’lOO" 0pCosts”50'' Ublss'200' Opteds'O' Boughts'O” YearAvallables'2’ /> 

<Values Num=’ll* Avall=’True' 0 D=’l' Type=’r Subtype="2’ lnvent="0" MaxAcq»’25* AcqCost=*125" 0pCost=’50" Utlls=’lS0" 0pted = "0‘ Boughts’O" Y«arAvallable=*4" /> 

</Weapons> 

</TempoYear> 

</Tempo> 


Figure 12. TEMPO Version 3 XML Log Lile-Last Year 
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This second feature required a major update to the existing TEMPO.ATL. 
Previously, an automatic play feature was unnecessary. In order to execute automatic 
strategies, TEMPO.ATL required; 

1. Capture of the current year’s environment 

2. Capture of the current year’s weapon availabilities 

3. Capture of intelligence data, if purchased 

4. An algorithm to analyze the captured data 

5. An algorithm to provide the weapon operation list and weapon purchase 
list 

6. A method to commit the weapon operation and purchase lists 

Items 1-3 were implemented by developing a turn-by-turn ‘interface file.’ Items 4 
and 5 were implemented by executing the specific rule-based strategies (discussed in the 
‘MATLAB Strategy Package Development’ section). Item 6 was implemented by 
designing TEMPO to read the interface file data and executing the weapon allocation 
decisions automatically. Visual Basic.NET was selected as the desired development 
environment for the TEMPO update (hence TEMPO.NET) because user interfaces can be 
quickly prototyped and it is a versatile, yet simple language to use. MATLAB was used 
to execute the rule-based strategies since it is a software language specifically designed to 
perform mathematical functions. However, using two different software development 
environments presented some unique challenges. 

B. DEVELOPMENT CHALLENGES 

Interfacing data between MATLAB and Visual Basic.NET development 
environments presented the most difficult challenge. Due to a lack of programming 
experience, the simplest method—transferring weapon data through reading and writing 
of plain text files—was chosen. Eor any given turn (i.e., year of play) the overall 
approach is the same. Eirst, TEMPO.NET creates a plain text file—tempo.ini’—listing 
which strategy the user requires. This is a simple text file that the MATLAB executable 

application eventually requires to execute the appropriate strategy. Next, TEMPO.NET 
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creates a plain text interfaee file—‘tempo.tmp’—^with the year’s environment and 
weapon availability data, and intelligenee purehased, if applicable. TEMPO.NET then 
calls a previously compiled MATEAB standalone exeeutable program 
(‘runStrategyExe.exe’) to read the data within these two plain text files, execute the 


strategy, and output the strategy results into a similar plain text file—‘tempo.str.’ Eigure 
13 displays a sample ‘tempo.tmp’ file from TEMPO.NET read by MATEAB. 



Eigure 13. 'Tempo.tmp' 


‘runStrategyExe.exe’ executes the strategy and writes a new text file (‘tempo.str’) 
to the same directory with the appropriate ‘operate’ and ‘buy’ lists. Eigure 14 displays the 
MATEAB-modified input text file, ‘tempo.str’. 



Eigure 14. 'Tempo.str' 
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Once complete, TEMPO.NET then imports this information back into the 
appropriate text boxes and the commits the weapon and intelligence allocation. This 
process repeats for every year until war occurs. The most difficult challenge involved 
timing TEMPO.NET to allow time for the MATEAB strategy to execute without 
overwriting files in the process. In order to do so, multi-thread execution was added to 
TEMPO.NET. One thread executes game play while the other continuously monitors for 
the ‘input file.’ Due to this implementation, the execution of the game in automatic mode 
updates the user interface unpredictably, so it is difficult to watch a game being played. 
Although not ideal, this solution provided the necessary functionality to support a 
consistent information exchange between TEMPO.NET and the MATEAB executable 
program. A more elegant solution would have been to compile a Dynamic Eink Eibrary 
(DEE) containing all the MATEAB functions for Visual Basic to call. This could also 
improve performance, as the current solution takes a considerable amount of time to 
execute (approximately 7 seconds per turn). Unfortunately, lack of programming 
experience rendered the DEE option too time consuming. 

The second major challenge was verifying the MATEAB strategy execution 
performance. Verification was required in order to ensure proper results from the 
automated trials were captured. However, many possible situations could occur during 
each turn. The verification approach was to categorize all possible situations into a test 
case category. A manually edited text file was produced to represent each test case. The 
verification of a strategy is successful once it passes all test case categories. These test 
case categories are described in Section 0. 

Ultimately, integrating SEVER and TEMPO is the objective. Therefore, SEVER 
requirements were also considered during all model and strategy software updates and 
development. The rest of this chapter describes SEVER, TEMPO, an example 
implementation of SEVER, and preliminary application of SEVER to TEMPO, to 
enhance the reader’s understanding of the SEVER algorithm and its application. 
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C. TEMPO.NET DEVELOPMENT 

These considerations constitute the basis for this publication. They are entrance 
criteria (essentially, requirements) that must be met prior to embarking on SEVER’s 
software development. 

• TEMPO—Include method to perform automated data entry. 

• TEMPO—Speed up player data input for more efficient human trials 

• TEMPO—Capture tum-by-turn data to evaluate real-time decision 
impacts (necessary for future SEVER implementation) 

• MATEAB—Analyze large numbers of game performance logs by 
decomposing performance measures to present those measures statistically 
based on the implemented strategy. 

Eigures 15 through 22 display the various features of TEMPO Version 3. These 
figures are intended to be self-explanatory. 



Eigure 15. TEMPO Version 3 Configuration Setup 
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Figure 16. TEMPO Version 3 Startup Dialog Box 


FSe View Tods About 



Figure 17. TEMPO Version 3 Main User Interfaee 
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Figure 18. TEMPO Version 3 Strategy Selection Dialog Box 



Figure 19. TEMPO Version 3 Strategy Menu Options 
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Figure 20. TEMPO Version 3 Analysis Menu Options 



Figure 21. TEMPO Version 3 Caleulated Values 
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Figure 22. TEMPO Version 3 Seore Display 


D. MATLAB STRATEGY PACKAGE DEVELOPMENT 

Rule-based strategies were necessary to automate the TEMPO execution. These 
strategies are also necessary to evaluate SEVER. Eour main strategy types were derived. 
They are; 

• Strategy 1—Random choice to Acquire/Operate 

• Strategy 2—^Acquire/Operate based on Pwar 

• Strategy 3—Concentrate on one type of force 

• Strategy 4—Invest in decisions based on INTEE gathered 

Strategy types 1 and 3 contain sub-strategies that the user can further tailor. Eor 
example, in strategy type 1, there are two possibilities to choose randomly how many 
weapons to operate and buy. The first establishes a random number for all weapons 
available. The second attempts to establish a random number for all weapons and also use 
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the entire available budget for that year. Strategy type 3 ean be tailored to foeus on 
operating/buying either offensive or defensive forees. In other eases, it ean be tailored to 
prioritize investing in eaeh partieular weapon system based on user preferenee. Thus, 
Strategy type 3 eontains many strategies within, as eaeh possibility is a permutation of a 
user-defined priority. 

Onee SEVER, TEMPO, and all interfaees were well understood, development 
followed. The first ehallenge was determining whieh software paekages to use. The 
eurrent version of TEMPO is a C++ ATE version. In order to input and extraet data 
dynamieally (i.e., turn by turn) modifying this eode was required. However, the 
spreadsheet user interfaee, although a vast improvement over the eommand eonsole text- 
based user interfaee, still laeked simple presentation that a noviee user eould quiekly 
learn to use. To improve both the user interfaee, and eontrol of the eode, the entire 
TEMPO model was ported to Visual Basio.NET. All funotions were transferred and muoh 
testing was performed to ensure the porting suooessfully oaptured all oritioal features 
available in the C++ ATE version. 

In order to exeoute strategies, MATEAB was seleoted beoause of its oapability to 
perform mathematioal and numerioal manipulation and analysis. It eontains many 
toolboxes to use for analysis and strategy oomputation. One of interest is the Markov 
Deoision Prooesses toolbox. Euture researoh eould inoorporate funotions of these 
toolboxes to improve on the strategies provided within this thesis or generate new 
strategies to test against TEMPO’S ooevolutionary eode. 

1. Strategy Development 

The work in this thesis involved updating the existing model to inoorporate user- 
defined strategies, oreating those strategies, and providing a oapability for the updated 
TEMPO to aooept these strategies. In order to test strategies, it was also required to 
update the TEMPO model to provide the oapability to perform a user-defined number of 
automatio games (or trials). Data from these trials must be able to be oaptured and 
statistioally evaluated. 
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To evaluate performanee of eaeh strategy, the strategies must automatieally 
integrate with TEMPO. Six different strategies were eoded using the MAThematies 
LABoratory (MATLAB)™ programming language. A eustom MATLAB funetion was 
eompiled using the MATLAB eompiler as a standalone exeeutable file 
(‘runStrategyExe.exe’). During an automatie game, TEMPO ealls this funetion prior to 
eaeh turn rather than waiting for human input. On the TEMPO side, during initial 
automatie game setup, the user is able to seleet the type of strategy and the number of 
games played (“trials”) desired. The interfaee between TEMPO and MATLAB oeeurs via 
three different text files—one to output the human and eomputer environments from 
TEMPO, a seeond to identify to the strategy exeeutable program whieh strategy was 
requested by the user, and a third to import into TEMPO the weapon alloeation ehoiees 
exeeuted by the given strategy. These files are named ‘tempo.tmp,’ ‘tempo.ini,’ and 
‘tempo.str,’ respeetively. 

Onee all trials have eompleted, a post-proeessing tool is used to eapture 
performanee data for the strategy. This post-proeessing applieation, 
‘AnalyzeStrategy.exe,’ was developed to eapture and present the data assoeiated with all 
the trials run, also using the MATLAB™ programming language. The data is read from 
the XML log files ereated by TEMPO, parsed and sorted into weapon alloeation variables 
for human and eomputer, separately, and output into the MATLAB workspaee. Speeifie 
plots relating to the performanee measures required (see next seetion) are displayed to 
visually eompare strategies. 

A third MATLAB standalone exeeutable file was developed to interaet with the 
user in manual mode on a turn by turn basis. This file, ‘graphStrategy.exe’ was originally 
developed to test that eaeh strategy exeeuted by ‘runStrategyExe.exe’ eorreetly exeeuted 
(i.e., applieable budget was used, and proper weapons were operated and purehased based 
on those available). Through subsequent debugging efforts, the tool ean now be used by 
the player to graphieally display the results of the past turn's strategy exeeution. 

The strategies eonsidered to test are deseribed below in algorithmie form. All 

strategies assume offensive and defensive intelligenee is always purehased, however, for 

this thesis, the only strategy to use this information is the ‘Intelligenee’ strategy. Also, a 
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strategy does not neeessarily use the entire budget available eaeh turn. Due to the rules of 
TEMPO, this unused budget is lost for eaeh subsequent turn. The strategies that do use 
the entire budget are deseribed as sueh. The strategies are; 

• True Random—^Alloeation determined randomly 

• Smart Random—^Alloeation determined randomly until total possible 
budget available is used 

• Probability of War—^Alloeation based on probability of war. 

• Priority—^Alloeation based on eoneentration of forees of a user-seleeted 
order or priority (i.e, Offense A, then Offense B, then Defense A, and 
lastly Defense B) until the entire budget is used. 

• Weapon Type (Offensive or Defensive)—Alloeation based on offensive or 
defensive forees only. Only those weapon types are operated and bought. 

• Intelligenee—^Alloeation based on a martingale-style rule to foreeast 
future eomputer weapon ehoiees. 

All strategies are exeeuted through a pre-proeessing funetion, 
‘runStrategyExe.m.’ This file aeeepts two inputs: 1. the ‘output file’ direetory loeation, 
and 2. the sub-funetion direetory to eall those sub-funetions as required. Eor every 
strategy, this funetion: 

1. Cheeks for an existing version of ‘tempo.str’ in the interfaee direetory and 
‘tempo.mod’ in the temporary write direetory. Deletes eaeh, if applieable. 

2. Parses ‘tempo.tmp’ from the interfaee direetory into the appropriate 
MATEAB workspaee variables. 

3. Calls the strategy requested by the strategy initialization file, ‘tempo.ini.’ 

4. — Executes appropriate strategy as defined below — 

5. Writes the ‘tempo.str’ file to a temporary direetory within the interfaee 
direetory from the MATEAB workspaee variables ‘buyEist’ and ‘opEist.’ 

6. Writes the ‘tempo.str’ file to the interfaee direetory. 
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To ensure eonsistency between the many strategies, a eommon MATLAB 
strategy-eall funetion strueture was established. The general funetion strueture for all 
strategies is: 

[intelList,opList,buyList] = A_5'trate’gyAame(weaponData,pWar,availBudget,oppData) 

In some eases, additional information was required based on the type of strategy. 
This information is passed first. For example, the ‘priority’ strategy required the priority 
of weapons in addition to the other input data. Therefore, its funetion strueture is: 

[intelList,opList,buyList] = A_priority(priority,weaponData,pWar,availBudget,oppData) 

The data on the right side of the assignment eontains all parsed variables passed 
via ‘runStrategyExe.m.’ Data on the left side are the outputs of the strategy algorithm, 
written by ‘runStrategyExe.m’ into ‘tempo.str.’ Some variables are not used throughout 
the strategy exeeution; however, this eommon format allows for future eustomizing with 
minimal updates required to the ‘runStrategyExe.m’ funetions. 

Please note, text below in bold indieates the speeifie funetion within the TEMPO- 
SEVER applieation-speeifie funetional deeomposition satisfied by the partieular step in 
the strategy algorithm. The TEMPO-SEVER applieation-speeifie funetional 
deeomposition is presented in Appendix B for information. 

a. Strategy #la: True Random 

(1) Deseription. The simplest (to implement) of all potential 
strategies, the ‘True Random’ strategy assigns values for the buy and op lists generated 
using a uniform random number generator. The algorithm ereates a 12 x 2 matrix of 
random numbers between 0 and 1. It then filters ‘available’ weapons by multiplying the 
randomly generated 12 x 2 matrix with the binary ‘available’ veetor. Simultaneously, it 
multiplies the matrix by the inventory and maximum aequisition-able values as reported 
by TEMPO.NET. These values must range from zero to the number of available 
‘inventory’ and ‘maximum aequisition-able’ weapons. The end result is a 12 x 2 matrix. 
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where one eolumn is the ‘opList’ and one is the ‘buyList,’ rounded to the nearest integer 
ranging from 0 to ‘invent’ or ‘maxAeq,’ respeetively. 


(2) Algorithm. 

1. Generate 12 x 2 matrix of random numbers to serve as ‘opList’ 
and ‘buyList.’ (1.3 Develop Information) 

2. Generate ‘opList’ and ‘buyList’ 

a. Multiply this matrix by the binary veetor of weapons 
available. This filters out the unavailable weapons. 

b. Multiply this matrix by the inventory and maximum 
aequisition-able values for all weapons 

3. Cheek to ensure the random alloeations for all weapons is 
within the available budget. If not, regenerate random numbers 
in same order and exeeute steps 1 through 3 again. Loop until 
all alloeations are less than the available budget. (1.3 Develop 
Information) 

4. Provide results as ‘opList’ and ‘buyList’ to 
‘runStrategeExe.exe’ whieh outputs to the ‘tempo.str’ file. (1.6 
Provide Recommendations) 

5. TEMPO.NET then exeeutes the turn with those weapon 
alloeations. (2.1 Provide Defense, 2.2 Provide Offense) 

b. Strategy #lb: Smart Random 

(1) Description. This algorithm is considered ‘smart’ because it 
attempts to use as much budget as possible, while still randomly allocating weapon 
choices. In the case when the budget available is greater than the sum cost of operating 
and purchasing all available weapons, all units will be operated/ purchased. 

NOTE: Eor the first turn of any strategy (including ‘True 
Random’) in an automatic game, this strategy is invoked. This ensures the computer 
cannot falsely win based on a sub-optimal first turn weapon allocation (since all available 
weapons can be operated and purchased with the available budget). Instead, a tie always 
results and the next year begins. 
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The algorithm executes weapon allocation decisions in a pre¬ 
determined randomized order calculated by ‘sequenceOrder.m.’ It then randomly assigns 
the operate/purchase sequence so that for every application of this strategy a different 
order of weapons is allocated each turn. This ‘randomized order’ ensures a truly random 
allocation despite the looping algorithm structure. 

(2) Algorithm. 

1. Gather ‘current environment state’ variable values (weapons 
Available and associated cost/performance data, probability of 
war, available budget, and computer utils from last turn if 
intelligence was purchased). (1.1 Gather Data) 

2. Determine, randomly, which order weapon systems will be 
operated (ref ‘sequenceOrder.m’). (1.3 Develop Information) 

3. Generate this list of random numbers within ranges of 
inventory available (for weapon operation) and maximum 
acquisition-able (for weapon purchase) for all available units, 
respectively in the order determined from ‘sequenceOrder.m.’ 

4. Buy/Operate weapons one at a time, in sequence and random 
numbers previously generated, until the available budget is 
used (always operating first then buying for each weapon in the 
order determined by ‘sequenceOrder.m’). 

5. Provide results as ‘opList’ and ‘buyList’ to 
‘runStrategeExe.exe’ which outputs to the ‘tempo.str’ file. (1.6 
Provide Recommendations) 

6. TEMPO.NET then executes the turn with those weapon 
allocations. (2.1 Provide Defense, 2.2 Provide Offense) 

c. Strategy #2: Probability of War 

(1) Description. As Pwar increases, weapon allocation shifts from 
acquire to operate for the highest performing weapons. This value is pre-set as the Pwar 
threshold. Currently, ‘threshold’ is not settable from the TEMPO.NET interface similar to 
how the priority and weapon type strategies require additional user input. 
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(2) Algorithm. 


1. Gather ‘current environment state’ variable values (weapons 
available, associated cost/performance data, probability of war, 
available budget, and computer utils from last turn if 
intelligence was purchased). (1.1 Gather Data) 

2. Determine weapon allocation based on a threshold probability 
of war occurring. (1.3 Develop Information) 

3. Establish performance rankings for all available weapons. 
Operation and acquisition of a single weapon are treated as two 
separate weapons since they have two separate costs. 
Performance is based solely on the ratio of Util/$ (ref 
‘rank.m’). 

4. Establish probability of war threshold value to switch between 
operating existing forces and increasing force levels. 

5. If the probability of war is less than the threshold, buy or 
operate units (using ‘activateWeapon.m’) depending on the 
performance rankings established in 3. 

6. Continue operating if probability of war threshold is greater 
than threshold value, or operate/buy, depending on 
performance rank, until all available budget is used if 
probability of war threshold is less than threshold value. 

7. Provide results as ‘opEist’ and ‘buyEist’ to 
‘runStrategeExe.exe’ which outputs to the ‘tempo.str’ file. (1.6 
Provide Recommendations) 

8. TEMPO.NET then executes the turn with those weapon 
allocations. (2.1 Provide Defense, 2.2 Provide Offense) 

9. Repeat from Step 1 for next year. 
d. Strategy #3: Priority 

(1) Description. The user establishes a weapon ‘order of 
preference’: i.e., (OA, then OB, then DA, then DB). Based on available weapons, the 
strategy always operates, then buys the maximum amount always considering this order 
of preference until the available budget is depleted. 
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(2) Algorithm 


1. Gather ‘current environment state’ variable values (weapons 
available and associated cost/performance data, probability of 
war, available budget, and computer utils from last turn if 
intelligence was purchased). (1.1 Gather Data) 

2. Determine weapon allocation based on user-defined priority of 
weapons to operate and/or buy. (1.3 Develop Information) 

3. Convert priority user input strings into binary ID’s (i.e., OA = 
[0,0]; DA = [1,0]; OB = [1,0]; DB = [1,1]) 

4. For all available weapons, operate first, and then buy the 
highest priority weapon. Weapon allocation execution occurs 
by weapon type order (i.e., OAl, OA2, then OA3, if all are 
currently available). 

5. Continue operating, then buying weapons until no more budget 
available (using ‘activateWeapon.m’) 

6. Provide results as ‘opList’ and ‘buyList’ to 
‘runStrategeExe.exe’ which outputs to the ‘tempo.str’ file. (1.6 
Provide Recommendations) 

7. TEMPO.NET then executes the turn with those weapon 
allocations. (2.1 Provide Defense, 2.2 Provide Offense) 

e. Strategy #4: Weapon Type (Offensive or Defensive) 

(1) Description. Selection of operate and acquire is focused on 
one type of offensive weapon system (Offensive or Defensive). This algorithm calculates 
the best performing weapons (highest Util/$ ratio) and operates them, then purchases, 
based on the priority given to either Offensive, or Defensive weapons. 

(2) Algorithm. 

1. Gather ‘current environment state’ variable values (weapons 
Available and associated cost/performance data, probability of 
war, available budget, and computer utils from last turn if 
intelligence was purchased). (1.1 Gather Data) 
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2. Determine weapon alloeation based on user-defined preferenee 
for ‘offense’ first, or ‘defense’ first. (1.3 Develop 
Information) 

3. Sort all available weapons into Offensive and Defensive 
weapons. 

4. Caleulate performanee measure (Util/$) for all available 
weapons using ‘oaleEffRatio.m.’ 

5. Rank operating and purehasing offensive weapons by 
previously ealeulated performanee measures. 

6. Rank operating and purehasing defensive weapons by 
previously ealeulated performanee measures. 

7. Operate, or buy preferred weapon type in order of performanee 
rank. Other weapon preferenee never gets operated. 

8. Provide results as ‘opList’ and ‘buyList’ to 
‘runStrategeExe.exe’ whieh outputs to the ‘tempo.str’ file. (1.6 
Provide Recommendations) 

9. TEMPO.NET then exeeutes the turn with those weapon 
alloeations. (2.1 Provide Defense, 2.2 Provide Offense) 

/. Strategy #5: Intelligence 

(1) Deseription. Determine weapon alloeation based on 
martingale theory. Paraphrasing Doob (1935), a sequenee is ealled a martingale if eaeh 
sample, Xn, has an expeetation, and if for m < n the expeeted value of Xn given the past up 
to time m is Xm. That is, the predieted sample’s expeeted value is the previous sample’s 
value (Doob, 1935). Thus, this algorithm bases the expeeted value of opponent utils for 
the future turn on the utils reported by intelligenee. Then, this rule extrapolates a new util 
value for the eomputer based on a linear sealing faetor. The sealing faetor is ealled the 
‘buffer’ and is required as an input to this algorithm. If the ‘buffer’ is eurrently eoded as 
0.10, then the expeeted value of opponent utils for the future turn are foreeasted at 10% 
above the utils from the last turn. 

Based on both the rank of performanee measures for eaeh weapon 


type along with the number of utils foreeasted, alloeate weapons to eover foreeasted utils 
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until all the available budget has been used. (If not all available budget is used and all 
foreeasted utils are operated, eontinue buying the highest performing offensive weapons. 
If the entire budget is used prior to eompletely operating all forecasted utils, the order of 
operation is DA, DB, OA, OB - the static order is a current limitation of the strategy) 

(2) Algorithm. 

1. Gather ‘current environment state’ variable values (weapons 
Available and associated cost/performance data, probability of 
war, available budget, and computer utils from last turn if 
intelligence was purchased). (1.1 Gather Data) 

2. Allocate weapons based on a forecast of the number of 
opponent Utils using the ‘buffer.’ (1.3 Develop Information) 

3. Calculate the forecasted opponent’s utils. (Note: these values 
for each weapon type are calculated previously to this function 
call in ‘runStrategyExe.exe’ and are passed as an input.) 

4. Calculate performance measure (Util/$*number available) for 
all available weapons. This is the utilRatio. 

5. Filter the utilRatio, inventory available, and maximum 
available to purchase into separate variables based on weapon 
type (i.e., OA, DA, OB, DB). 

6. Calculate number of Utils required based on the forecasted 
opponent’s data. 

7. Defend up to number of forecasted utils, then operate highest 
ranking weapons with remaining budget. 

8. Provide results as ‘opList’ and ‘buyList’ to 
‘runStrategeExe.exe’ which outputs to the ‘tempo.str’ file. (1.6 
Provide Recommendations) 

9. TEMPO.NET then executes the turn with those weapon 
allocations. (2.1 Provide Defense, 2.2 Provide Offense) 
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E. MATLAB ANALYSIS PACKAGE DEVELOPMENT 


The first step to generate this analysis paekage was to determine what was 
required to be verified. At this point in time, verifieation was only required for the 
strategy output. In future versions of the paekage, verifieation will inelude the analysis 
and assessment of SEVER. 

Assessment of SEVER would be foeused on a set of experiments eomparing 
several automated strategies against the eomputer. A set number of trials with eaeh 
strategy would be exeeuted and the outeome eaptured. SEVER would evaluate eaeh 
strategy as they exeeute. The hypothesis eould be that the predieted SEVER value 
ealeulations for eaeh turn eorrelate with the statistieal performanee results of eaeh 
strategy. Eurther researeh eould eombine the output of SEVER into an additional 
strategy. This thesis presents the analysis on six general strategies eoded in MATEAB. 
The analysis proeess ean be repeated when the time eomes to analyze the SEVER 
algorithm when exeeuting these six strategies. 

But how are the strategies verified to produee eonsistent, re-produeible buy and 
operate lists? This seetion foeuses on the testing proeess for eaeh strategy. 

1. Strategy Testing 

Verifieation of eaeh strategy involved passing all test ease eategories. The 
standard TEMPO.NET output file (‘tempo.tmp’) was manually edited to ereate that ease 
and the strategy under test was exeeuted. A graph of the strategy’s output for eaeh 
weapon for eaeh test ease was developed to prove verifieation. Table 1 summarizes the 
extreme eases tested for all strategies. If the results were eonsistent with those expeeted, 
under all eategories, the strategy was verified and a eheek was plaeed in the box for that 
ease. 


61 



Table 1. Test Case Category Matrix 


Weapons 

available 

Max Budget Available 

No Budget Available 


Enemy 

Purchased... 2 

No Intel 

Enemy 

Purchased... ^ 

No Intel 

All 

□ 

□ 

□ 

□ 

None 

□ 

□ 

□ 

□ 

OA 

□ 

□ 

□ 

□ 

OB 

□ 

□ 

□ 

□ 

Offensive 

□ 

□ 

□ 

□ 

DA 

□ 

□ 

□ 

□ 

DB 

□ 

□ 

□ 

□ 

Defensive 

□ 

□ 

□ 

□ 


The MATLAB standalone exeeutable ‘graphStrategy.exe’ eaptures a graphieal 
output of the strategy exeeuted for the latest input fde, ‘tempo.tmp.’ This exeeutable was 
run for each modified ‘tempo.tmp’ file for a total of 16, or 32, test runs depending on the 
strategy. The output of this function was an allocation graph focused on that particular 
strategy’s goal. Details on each strategy’s verification are located in Appendix C. A 
general description of the verification particulars of each strategy are described below. 

a. True Random 

This strategy is the simplest of all strategies. Verifying this function meant 
ensuring all available weapons, over a large number of ‘True Random’ executions would 
generate a uniform distribution of allocation between 0 and the total number of weapons 
available for operate and purchase for that turn (see Figure 23). The execution of this 
verification was to run, for each turn, a user-defined number of trials (100) ensuring that 
any value between 0 and the maximum (for both operating and purchasing all available 
weapons) was executed. No attention was paid to budget used, except to note that the 
strategy did not provide an allocation that would result in using more than the available 
budget. 


2 


This case only applies to strategies making use of the Intelligence gathered. 
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Figure 23. True Random Strategy Verification Graph 


63 

























































b. 


Smart Random 


This strategy built upon the previous strategy by randomly alloeating 
values until all budget was used. The verification for this strategy was focused on 
ensuring all budget was used until no more available weapons could be 
operated/purchased (see Figure 24). This was verified by performing a user-defined 
number of trials (100) for each turn. A bar plot (budget vs. trial) then displayed the results 
of each trial performed. If any budget remained that could have purchased or operated 
another weapon, that trial’s bar was flagged red. Otherwise, it was green. Thus, the 
strategy was verified to operate correctly when all bars were green. 
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Figure 24. Smart Random Strategy Verification Graph 
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(1) Probability of War, Priority, Weapon Type. Eaeh of these 
strategies was verified using the same method. The weapons available and their 
operate/purehase maximum quantities were reeorded. Given these values, for each 
strategy, the proper allocations for each weapon subtype (i.e., OAl, DB2, etc.) was 
recorded and coded into the ‘graphStrategy.exe’ function. Next, the actual strategy was 
executed. Results were stored by the function. Finally, for each strategy separately, the 
graph displayed predicted and actual on a grouped bar chart side by side (see Figure 25). 
Many different scenarios were tested by modifying the ‘tempo.tmp’ file to create peculiar 
situations per Table 1. Once the actual results matched the predicted results for all 
scenarios attempted the strategy performance was successfully verified. 
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Figure 25. Probability of War, Priority, and Weapon Type Strategy Verification Graph 

(2) Intelligence. This strategy was verified using the method 
described above for Probability of War, Priority, and Weapon Type except for verifying 
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one additional feature; the use of the opposition’s data when purchasing intel. Both the 
‘Weapons’ and ‘OppData’ sections within ‘tempo.tmp’ were modified to capture each 
different case per Table 1. This resulted in more trials performed in order to verify proper 
strategy execution, but the verification methodology remained consistent. Also, to 
facilitate verification, the graphical output also displayed the intelligence reported for 
quick reference between the weapons operated/purchased and the intelligence values 
captured. See Figure 26 for the intelligence verification graph. 


Weapons Opetatad - Budget Left t2S 



Weapons Bought • Budget Left $26 



Figure 26. Intelligence Strategy Verification Graph 

2. Measures of Performance 

These consist of two categories, “per game” and “per turn” measures of 
performance. Ultimately, SEVER helps evaluate a given strategy’s effectiveness at 
meeting the top-level function “Win game.” Flowever, because the game has random 
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variables (i.e., available budget) assoeiated that eould potentially provide an advantage to 
either the player, or eomputer, a more detailed analysis provides insight into the 
shorteomings of implementing SEVER. 

In this implementation, eaeh strategy was analyzed per eaeh weapon and per total 
weapons. These measures were also filtered into a per-game outeome and per-turn 
outeome. This analysis is exeeuted at the lowest level resolution possible by the game. 
Therefore, for future evaluation using SEVER, it will be possible to answer any potential 
questions of a partieular game or turn. Eurther researeh eould build upon these 
performanee measures to somehow ineorporate these measures as a eolleetive output for 
use by SEVER. 

Per Turn: 

• Outeome of eaeh turn (1 for win, 0 for lose) 

• Total performanee at end of turn (# of util differenee) 

• Weapon-type performanee at end of turn (# util differenee) 

• Total Effioieney of strategy (# Wasted Utils) 

• Weapon-type effieieney of strategy (# Wasted Utils for eaeh weapon type) 

Per Game: 

• Outeome of eaeh game (1 for win, 0 for lose) 

• Performanee at end of game (# of util differenee) 

The performanee measures deseribed above are those presented in this thesis. 
However, beeause all relevant environment and weapon alloeation data for both player 
and eomputer is eaptured turn by turn in the XME performanee log files, it is possible to 
perform post-proeessing to glean any desired performanee measures at time of analysis. 

The first researeh questions state: “Can TEMPO be updated and modified to 
aeeommodate SEVER? What is required? How ean this be aehieved?” In order to 
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evaluate these questions, the development of the TEMPO Version 3 software package 
commenced. Out of this development effort, more focused questions were derived. These 
questions include: 

• Are there consistent, predictable patterns for which weapons the computer 
is likely to operate/buy? 

• Is the TEMPO-reported probability of war each turn truly accurate? 

• Is the computer smart enough to change its strategy significantly given a 
certain number of trials against a rule-based player? 

The answers to these questions will help determine if TEMPO Version 3, as it 
stands, is adequate to evaluate SEVER. These questions will be answered in Section G. 

Results and Discussion 

F. INTEGRATING SEVER AND TEMPO 

This section describes the general process attempted to apply SEVER to TEMPO. 
Hypothetical, but realistic cases are presented to establish the envisioned information 
exchange between TEMPO and SEVER. Work in this thesis does not reflect a complete 
integration of SEVER and TEMPO since SEVER’s notion of ‘quality’ was not captured. 
However, TEMPO.NET does include features to facilitate TEMPO-SEVER integration 
once ‘quality’ can be properly represented. These are the current SEVER place-holder 
features: 

• Per weapon graphical ‘stop-light’ icons—Display value as calculated by 
SEVER as ‘high,’ ‘medium,’ or low depending on the user weapon 
decisions of ‘operate’ and ‘buy’ for each that weapon (see Eigure 27). 

• Per weapon-type interactive track bar—^Allows user to select a 
weight/priority for each weapon type that allows SEVER to dynamically 
calculate the overall value for the displayed weapon allocations (see 
Eigure 27). 

• Per turn total SEVER output—Displays the probability of ‘Win game’ 
given ‘War Occurs’ with the allocations listed at the time. Updates in real¬ 
time when allocations are changed to allow the user to affect the 
probability of success ‘on the fly.’ 
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Figure 27. SEVER Place-Holder Eeatures 


G. RESULTS AND DISCUSSION 

All the results stated in this section focus on answering the original thesis 
questions; Can TEMPO be updated and modified to accommodate SEVER? What is 
required? How can this be achieved? As mentioned at the end of Section 0, detailed 
questions were derived to help answer these questions. The detailed questions are; 

1. Are there consistent, predictable patterns for which weapons the computer 
is likely to operate/buy? 

2. Is the TEMPO-reported probability of war each turn truly accurate? 

3. Is the computer smart enough to change its strategy significantly given a 
certain number of trials against a rule-based player? 

In order to answer these questions, TEMPO Version 3 commenced so that the 
applicable data could be captured. This software development effort included the 
following updates to original code; 
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• Created an automatie game-play feature that allows a defined number of 
games to be played with partieular strategies 

• Developed the rule-based resouree alloeation strategies 

• Updated the GUI to inelude SEVER outputs 

• Condueted many games (trials) to prove the output of pWar is aeeurate 
and eharaeterize performanee of the eomputer opponent 

The rule-based strategies were verified and eompared against one another to 
demonstrate the statistieal evaluating eapability of the MATEAB software paekage. Sinee 
all data is eaptured within MATEAB workspaee variables, any desired data analysis ean 
be performed either during, or after all trials and strategies have exeeuted. This allows the 
user to exeeute a partieular strategy for a defined number of trials and perform 
performanee analysis on those trials without having to see eaeh trial performed. TEMPO 
Version 3 allows a user-defined data set to be ereated. For previous TEMPO versions, 
eonstrueting sueh a dataset required the user to laboriously perform eaeh trial aeeording 
to eertain rules and extraet the data manually into a storage database. This method was 
extremely time eonsuming and prone to human error. 

After development of TEMPO Version 3 and verifieation of eaeh rule-based 
strategy, a data set of 1000 trials was eolleeted for eaeh of 10 sub-strategies. This data set 
was analyzed to answer the detailed questions stated earlier: Can TEMPO be updated and 
modified to accommodate SEVER? 

What is required? 

In order to evaluate this question, the TEMPO Version 3 software development 
effort was exeeuted. Three questions from the TEMPO Version 2 analysis whieh are 
important to the updated development: 

• Are there eonsistent, predietable patterns for whieh weapons the eomputer 
is likely to operate/buy? 

• Is the TEMPO-reported probability of war eaeh turn truly aeeurate? 

• Is it possible to present performanee statisties for eaeh strategy and 
eompare them? 
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To answer each of these three questions, a determination if TEMPO Version 2 
performance and game parameters could be captured and modeled for statistical analysis. 
By capturing data about each turn of each game including both player’s decisions, 
environmental parameters, and outcome for every turn of every game, the data set would 
be complete to answer the three questions posed above. Based on the work performed in 
this thesis, it was determined that TEMPO Version 2 could be updated in such a way to 
capture all the required data. 

How can this be achieved? 

TEMPO Version 3 was developed with two premises: creating particular strategy 
datasets and capturing all decision data, environmental data, and outcome data for every 
turn of every game for any strategy employed. In order to create this data, an interface 
between TEMPO and MATEAB was constructed. Automatically, strategies were 
executed by MATEAB and imported back into TEMPO. Also, the normal (manual) 
game-play mode was retained with the same data capture and analysis features as for 
automatic mode. Eurther, a flexible software architecture to allow for future strategy 
development was implemented since incorporating SEVER would be performed in a 
phased approach. 

In order to capture the data, the TEMPO Version 2 XME log files were exploited. 
These log files contain all the required information for every turn of every game. A 
MATEAB function was created to parse the XME log files output by TEMPO and store 
the data permanently in MATEAB workspaces. Post-processing is now possible on all 
existing log files, either game-by-game, or dataset-by-dataset. Also, during manual game- 
play a feature was added to facilitate the player’s decision making by exploiting this 
MATEAB XME post-processing analysis package and analyzing the interfacing 
‘tempo.str’ file. Regarding SEVER, this feature is very important. SEVER attempts to 
score a particular decision in real-time. Therefore, TEMPO Version 3 must be able to 
evaluate data tum-by-turn as well as after a game is over so that this data can be 
compared against the SEVER-predicted data for both turn-by-turn and game-by-game 
evaluation of SEVER. Also, this feature could be used for future decision-making 
research as alluded to earlier. 
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Understanding what is required to accommodate SEVER into TEMPO Version 2 
involves understanding the questions detailed below: 

Are there consistent, predictable patterns for which weapons the computer is 
likely to operate/buy ? 

In 2005, Johnson, Melich, et al. performed work to adapt the computer strategy by 
implementing an iterative, coevolutionary learning approach whereby many different 
decision rules were used by two computer players against each other. Poor performers 
were eliminated and good performers were retained to give rise to new decision rule sets 
by “copying,” “mutation,” and “crossover.” Eurther research could be conducted to 
implement the same effort with the existing strategies coded into the MATEAB software. 
Thus, the computer opponent does, indeed, operate and buy with consistent patterns. 
Characterizing these patterns will help to evaluate the effectiveness of SEVER. 

Is the TEMPO-reported probability of war each turn truly accurate? 

The true calculation of the “probability of war” value displayed by TEMPO 
represents the number of games with war in year n divided by the number of games 
reaching year n. After all trials were performed, the data were captured to determine after 
exactly which years war occurs, given the particular year each trial reaches. The resulting 
calculation is shown in Eigure 28. It is evident that there is an anomaly with the 
‘probability of war’ output by TEMPO Version 2, and also TEMPO Version 3 (as this 
function was directly ported from the Version 2 code). This value is planned to be used 
by SEVER and therefore must be accurate in order to evaluate the output from SEVER 
properly. 

Is it possible to present performance statistics for each strategy and compare 

them ? 

Yes, this is possible since the existing software contains a feature that records the 
outcome of each game with all the environmental data and allocation decision data for 
both players. These performance statistics will be used as the baseline statistics against 
which the predicted SEVER output is evaluated. 
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As an example, Figure 29 displays the performance of each strategy based on 
number of TNO utils. The red line represents the average value of all data. The blue box 
represents 95% of the data. SEVER would not be used to predict the actual TNO utils as 
displayed in the chart, but rather the probability of winning the game for a given strategy 
as demonstrated by the boxes on the plot. 



Eigure 28. Flistogram of'War Occuring' Year 
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Box Pto(s of Player Net Utils for All Strategies 



^rat«gies 

Figure 29. Box Plots of Player TNO utils when War Occurs (per Strategy) 

Regarding SEVER, one important question still must be answered to properly 
integrate with TEMPO—how can the SEVER-defined quality be properly represented by 
the TEMPO variables? The answer to this question is the last item required before 
TEMPO can be used to evaluate SEVER as a strategy-scoring and prediction algorithm. 
However, all efforts performed to this point indicate that TEMPO can, and should be 
used to accommodate SEVER. The TEMPO Version 3 software includes many new 
features that are conducive for SEVER to function as developed by Eangford (2006). 

Now that datasets can be captured and stored in MATEAB workspaces, all 
features of the TEMPO version 2 log files can be used to characterize game parameters 
statistically. Eor example, computer decisions for any given turn can be estimated. Also, 
impacts of a player’s strategy can be analyzed for any turn in any game. This allows 
SEVER’s quality variable for a given decision to be successfully modeled. 
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H. CONCLUSIONS AND RECOMMENDATIONS 


Evaluation of SEVER is now possible using TEMPO Version 3. TEMPO Version 
3 allows a defined number of trials to be eondueted. These results ean be plotted in box 
plot format. There are dedieated areas within the GUI to ineorporate SEVER outputs. 
Any type of strategy ean be eoded and exeeuted on the TEMPO Version 3 platform. Post- 
proeessing of any game variable is available within the GUI. Post-proeessing of any turn 
weapon alloeation is available, but not yet within the GUI. These new features 
signifieantly enhanee and automate the type of analysis that ean be performed turn-by¬ 
turn, game-by-game, and strategy-by-strategy. These enhaneements now allow the 
predieted strategy value determination of SEVER to be eompared against aetual data. 

In summary, the work presented in this thesis eonfirms the answers to the researeh 
questions; 

1. Can TEMPO be updated and modified to aoeommodate SEVER? Yes. 

2. What is required? Eurther questions were derived. In addition to the 
answers to these questions, it was required to update TEMPO Version 2 to 
ereate strategy datasets and eapture all data. 

a. Are there eonsistent, predietable patterns for whieh 
weapons the eomputer is likely to operate/buy? Yes, there are. 

b. Is the TEMPO-reported probability of war each turn truly 
accurate? At this time, it has not been eonfirmed that these values 
are truly aeeurate. Evaluation of this aspeet of the TEMPO Version 
3 VB.NET eode is required for future integration and evaluation of 
SEVER. 

e. Is it possible to present performanee statisties for eaeh 
strategy and eompare them? Yes. This is demonstrated in Eigure 
29. 

3. How can this be achieved? Eurther questions were derived. Answers to 
these questions summarize how it is possible to aoeommodate SEVER into 
TEMPO Version 2. 

a. How should TEMPO Version 2 be modified? Create a oapability to 
run strategies automatioally and store all environmental data, 
player and opponent data, and per-game outoome data. Design in 
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the flexibility to add strategies to the software to further 
characterize the computer’s performance so that the SEVER 
representation of quality can be incorporated. Currently, this 
feature is only implemented via MATEAB text-based functions 
run on a particular MS Windows Explorer folder containing the 
XME log files. There is a placeholder for this feature, as shown in 
Eigure 20. (ref. ‘The Game’). Also, allow turn-by-turn analysis to 
be executed since SEVER can be used to evaluate a single turn’s 
weapon allocation strategy along with a particular game’s overall 
allocation strategy. This feature is also implemented via 
MATEAB, but can be activated through the TEMPO Version 3 
interface as shown in Eigure 20 (ref ‘This Turn’) 

b. Can the TEMPO game be modified to evaluate game play resource 
allocation decisions while the game is played (i.e., in ‘real-time')? 
Yes, this is possible with the use of ‘tempo.str’ as the data for 
analysis rather than the XME game log file. As previously 
mentioned, this feature can be found in the software as shown in 
Eigure 20. 

Other than the required update to incorporate SEVER: characterize the TEMPO 
performance to evaluate quality, the author suggests some software improvements. The 
following software improvements are suggested to increase the time it takes the in-game 
strategy and analysis functions to execute: 

• Replace MATEAB standalone executable called from TEMPO.NET with 
a MATEAB DEE containing all associated functions and code directly 
into TEMPO.NET. This would eliminate the continuous search loop 
required to scan for ‘input fdes.’ 

• Create the GUI for the MATEAB analysis code for the XME Eog file to 
streamline the post-processing and analysis of game log files. 

• Create a GUI for the MATEAB analysis code independent of the TEMPO 
GUI so that post-processing can be executed simply. 

• Perform true statistical analysis using the MATEAB Statistics Toolbox on 
the existing, and future, strategies. 

These suggestions are intended to improve upon the features and capabilities that 
already exist in TEMPO Version 3. The new features of TEMPO Version 3 are the first 
step in improving the human’s decision process. These features are an attempt to consider 
a high-level management resource allocation decision (in this case, budget) and provide a 
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total systems approach to collect, organize, and summarize the impacts in an automated 
fashion using today’s powerful computational processing. This saves the user time 
because he/she does not have to physically and mentally organize this information 
himself/herself Therefore, more effort can be spent on assessing a particular outcome’s 
consequence. Based on the user’s preference, select information can be accessed in real¬ 
time to allow quicker understanding of a given decision’s impacts just after it is executed. 
If successful, the incorporation of SEVER will further build on this capability by being 
able to predict the outcome prior to executing the decision. 

Ultimately, the goal is to automate a DoD resource allocation decision process 
based on SEVER-predicted outcomes and applicable presentation of the right information 
at the right time. Whether the decision is a “satisfaction” decision involving, strategically, 
how forecasted budget should be allocated toward future versus current military 
capabilities, or an “optimization” decision involving what the optimal tactical force 
structure is required in a certain forecasted conflict, the software tool presented in this 
research, along with the SEVER algorithm, can one day be adapted to present the 
required information at the right time to the eventual decision maker. 
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APPENDIX A. VALUE SYSTEMS ENGINEERING (GARY 

LANGEORD) 


Appendix A was prepared and written by Professor Gary Langford, my 
advisor at the Naval Postgraduate School. It summarizes the metrics of 
Value Systems Engineering, in so far as it relates to a framework that can 
be used to further the evaluation of SEVER as a game theoretic. The 
included references cite works that are integral to Value Systems 
Engineering which form the foundation of this thesis. 

This section formulates and applies a simple rootage for Value Systems 
Engineering (VSE). This conceptualization of VSE embodies the same general notions of 
systems, system elements, functions, performance, and quality, but redacts many of the 
fundamental approaches commonly used (Eangford & Huynh, 2007) in Systems 
Engineering. Eor example, the essence of a system is a set of elements that are either 
dependent or independent but interacting pairwise—temporally or physically—to achieve 
a purpose. Elements that only interact directly with other system elements are internal to 
the system. Elements that interact with both internal and external elements form the 
boundary of the system. We include the permanent and episodic interactions among 
elements of a system, systems of systems, and a family of systems. A system thus 
includes the lasting and occasional interactions, as well as emergent properties and 
behaviors. The interactions between elements effect transfer of energy, e.g., materiel, 
data, information, and services. The interactions can be cooperative or competitive in 
nature, and they can enhance or degrade the system value, which is defined below. 

A. FUNCTION 

We define the worth of a system (or product or service) in terms of the system 
functions, their performances, and their qualities (i.e., a Taguchi loss function). Eor 
example, a product shall provide a function with specified performance and a delimited 
level of quality. A function is an action performed by the system that is required to 
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achieve a system objeetive. System funetions may ehange and be added or deleted. The 
eoneepts of performanee and quality of a funetion will be elaborated in the following 
diseussion. 


B. VALUE 


Value (V) per function is defined as the ratio of performanee (P) to investment (I), 
the fundamental premise of Value Engineering (Miles, 1972). Value eompares what one 
reeeives with what one has invested. If there are two produets with faetually eomparable 
features offered for different priees, the value of the lower-prieed produet is higher than 
that of the other produet (Langford, 2006). The value of a system is measured by its 
worth (the aetual and expeeted use of a produet or serviee) relative to the investment 
made in obtaining the system. The system value may vary with time. To aeeount for 
additional investments made during the system lifeeyele, the investment ean also ehange 
with time. The Value Engineering equation, whieh relates performanee and eost, ean be 
rewritten to inelude their implied relationship to the same funetion(s). 

The system value, V(t), is given by 


v(0^f<') 


v ^(0 


(A.1) 


where 7^( t) is a funetion or non-linear summation of funetions that are performed by the 
system, P( t) is the performanee measure (units of energy) of the funetion(s) F(t) , I(t) 
is the investment (e.g., dollars or other equivalent eonvenienee of assets that are ‘at-risk’) 
and the time, t, measured relative to the onset of initial investment in the projeet. The 
investment ean be ineremental and summed to equal the lifeeyele investment or 
partitioned and equal to a unit or item investment. The units of V(t) ean be expressed in 
terms of energy divided by eost. We refer to the delineation of a funetion in terms of its 
performanee, and the quality of that performanee, as the triadie deeomposition of the 
funetionE(f). The summation in Equation (A.l) is simplified for the purposes of this 
diseussion, and thus shown over all funetions, performanees, and investments. 
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This construct of value is used by Value Engineers to answer the question, “Why 
does something increase the cost so much”? Value analysis is a problem-solving system 
that assists in the application of better approaches, alternative materials, appropriate 
processes, and in identifying the advantages of various suppliers. The point of Value 
Engineering is to be more effective in reducing costs without compromising on satisfying 
the customer. 


C. PERFORMANCE 


Performance indicates how well a funetion is performed by the system. 
Performanee is an objective measure of its related funetion. In this general construct and 
application for software piracy, quality refers to the consisteney of performance (or 
designated toleranee that signifies the deviation allocated to the performance 
requirement) in reference to the amount of pain or loss that results from the deviation as 
described by Taguchi (2005). 

The change in performance of a system element due to the transfer of energy from 
another element is equal to the work done. Performance is accomplished with reference 
to the cost/unit time as well as to the total time over which the performance occurs. 
Incorporating the variable of time and then factoring it results in expressing the value 
equation in terms of the metric of performance per rate of investment (e.g., spending). 


V(tfFlO = 


y P(0 

t 


(A.2) 


In essence, this formulation of Value Systems Engineering implies that functions 
result in capabilities; performances differentiate competing products; and quality affects 
the lifecyele eost of the product. Eor each function, there is at least one pair of 
requirements—a set of performanee requirements for eaeh function and a set of quality 
requirements for each performance requirement. 
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D. QUALITY 


The quality requirement indieates the variation and impaet of that variation on the 
performanee requirement of a funetion. Quality indieates how well a funetion is 
aeeomplished (through its performanee) by the system. It is a measure of the variation 
and impaets of the variation of the performanee requirement(s), or that performanee 
aehieved by the system, assoeiated with its related funetion. Additionally, quality is a 
measure of the loss due to the performanee of the system. The performanee requirement 
is measurable and testable. The quality requirement derives from the view of the system 
throughout its lifeeyole and eharaeterizes the system losses due to predetermined 
funetions, i.e., non-delivery of the system’s funetionality, or operations beyond the range 
of speeified performanee toleranees. A system funetion may thus have any number of 
performanee parameters and, likewise, several quality requirements assoeiated with a 
given level of performanee. 

E. WORTH 

Worth (W) is the use of a produet or serviee as represented by the funetions and 
their related funetional attributes—^performanee, quality, and investment. Multiplying 
Value from Equation (A.2) by Quality is defined as the Worth of F(t). Additionally, 
multiplying the numerator and denominator by P(t) defines the performanee metries and 
indieates that quality is a measure relative to performanee. 

Wit) = [V (Qz .,0. Qit)] = X *—1^] (A.3) 

^ lit)It t Pit) 

where Qit) is the quality (whieh ean be eonsidered as a toleranee assigned to P(t)). 
Stipulating the units ofQ(t) to be the same as that of/(t), determines the unit oiW(t) to 
be that of Pit), sinee F(t) is dimensionless. The summation in (A.3) is simplified for 
the purposes of this diseussion, and thus shown over all funetions, performanees, quality, 
investments and temporal notions. Equation (A.3) times likelihood is referred to as the 
Systems Engineering Value Equation with Risk (SEVER). Risk is diseussed in seetion g. 
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The interaction between elements transfers a measure of worth (defined as value 
convolved with loss) from one element to the other element of the pair of elements. We 
term the measure of the transferred worth the Worth Activation Function {WAF). 

F. WORTH ACTIVATION FUNCTION 

The WAF is a basis for analyzing a system. In control theory, a transfer function is 
a mathematical representation of the relation between the input and output of a system. A 
WAF between two elements of a system is defined to be the exchange of value between 
the two elements. The elements exchange energy and one measures performance. Value 
is that performance achieved for a given investment. This exchange necessarily assumes 
some measure of risk. Given risk, a WAF can thus be either a manifestation of the state, 
(or a change in state of a system) or a tool to evaluate differences between the state of a 
system and the state of another system or between the states of two systems in a system 
of systems. In essence, the WAF represents various impact(s) on the state(s) of a system. 
The WAF can be a nested hierarchy of WAFs, all related through the triadic 
decomposition of functions, performance, and quality. Depending on the value ascribed 
to each of the WAFs, the state(s) of the system(s) may be impacted to varying degrees. 
The result is that a small number of WAFs may be equivalent to a large number of 
irreducible WAFs. A small number of highly decomposable WAFs may be equivalent to a 
large number of irreducible Worth Activation functions. 

G. RISK 

Using the logic in from Lowrance (1976), Lewis (2006) defines simple risk as a 
function of three variables: threat, vulnerability, and damage. Replacing damage with 
worth, Langford and Homg (2007) capture risk through threat, vulnerability, and worth. 

An element e of a system is associated with a risk, Rg defined by 

Re = X^U^W^ = XX\-a^)WAF^ (A.4) 

where, threat, , is a set of harmful events that could impact the element; vulnerability, 

Ug is the probability that element e is degraded or fails in some specific way, if attacked; 
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worth activation function, WAF ^, results from a successful attaek on element e ; and 
suseeptibility, , is the likelihood that an asset will survive an attaek. WAF^ is given by 
Equation (A.3). It may be loss of produetivity, easualties, loss of eapital equipment, loss 
of time, or loss of dollars. Suseeptibility is the eomplement of vulnerability. 
Suseeptibility is loosely defined as the inability of a strategy to avoid being eountered in 
a game-eompetitive environment, whereas vulnerability is the inability of the strategy to 
withstand eountering eaused by the threat. Suseeptibility and vulnerability ean be 
measured by the probabilities of these events happening. Therefore, the probability of 
strategy surviving a game-eompetitive environment (strategy survivability) = 1 - 
Probability of the strategy being eountered (suseeptibility) x Probability of the system 
sueeumbing to the effeets of the strategy (vulnerability). 

Sinee an element in a system (or network) may be eonneeted to more than one 
element, the number of WAFs assoeiated with the element is the degree of the element. 
Subseribing to Mannai and Lewis (2007), we obtain the system risk, R , as 

n+m 

R = Y.^;(\-a,)gmF, (A.5) 

(=1 

in whieh n denotes the number of elements, m the number of links or WAFs, and 

Si denotes the degree of the 1“' element. As a result of the WAF between two elements. 
Cl and ^ 2 , at the moment of their interaetion for some elements, we have 


ITAF HAF 


R 


R 


(A.6) 


It is this expression in Equation (A.6) whieh forms the basis for understanding 

transaetions between elements (i.e., exehange of energy) that are independent and arms- 

length (e.g., buy-sell arrangement between unrelated parties) within a partieular system 

for whieh the Worth Aetivation Eunetion is defined (Langford et al. 2007). Equation 

(A.6) is also the basis for defining and evaluating eomplexity as well as interpreting and 

predieting emergent properties of systems. Complexity and emergenee are determined by 

numbers of elements, their Worth Aetivations Eunetions, and probability that the Worth 

Aetivation Eunetions will be at the Pareto-optimum value(s). 
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H. DECISIONS BASED/CAPTURED BY SEVER 


Decisions (from the perspective of the human game player) are represented in 
Equation (A.4) as the set of potential losses, i.e., a reduction in the quality of a follow-on 
decision for a given level of the performance achieved as a consequence of a former 
decision. For the high quality decisions there are also other losses that accrue, such as 
game-play length of time for the turn. From Equation (A.4), the losses from a decision 
can be represented as shown in Equation (A.7). 


Disruption Due To Low Quality Decision = 


m 

Pit) 


(A.7) 


These losses are typified by a loss function of a quadratic form for game- 
competitive environments. From Equation (A.7), the highest quality decision from the 
perspective of the human game player’s value chain is represented without the disruptive 
factor. Equation (A.8) as: 

HighQuality Decision = [Vit)^fio^Qit)] = 6(0 j 

^ lit)It t Pit) 


I. RAPID SYSTEMS ENGINEERING 


We apply the general methodology of Rapid Systems Engineering (Eangford, 
2006) to explore the Worth Activation Function’s general utility to characterize software 
piracy through its Stakeholder Analysis Methodology. Rapid System Engineering (RSE) 
is a scenario-driven approach that attempts to reduce the degree of uncertainty in 
predicting enterprise success by structuring and analyzing the interplay between 
alternative operational models, competitive strategies, and their resultant product 
alternatives. The RSE structure is a bottom-up, systematic, and highly iterative set of 
steps that marry competitive strategies to alternative operation’s requirements and 
conditions for stakeholder success. Applying RSE to software piracy presupposes that the 
four attributes of a software pirate’s successful business operation are satisfied. These 
attributes are first, the business value proposition articulated (the reason customers buy 
the pirate’s products and compensate the pirate. This could imply the software pirate is in 
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the business of making a profit of the labors of others or that the software pirate is 
satisfied through other means or viearious extensions. Seeond, the software pirates have 
identified market segment(s) or groupings of eustomers who are aggregated in a eommon 
distribution ehannel or some other segmentation that results in an eeonomy of distribution 
that is aeeeptable by the pirate. Third, the strueture, aetivities, and proeesses that 
eomprise the pirate’s praetiees that eontribute to the worth are defined. These inelude the 
value ehain and its logieal relationships of low-order separation. Fourth, the meehanies 
and venues of generating revenue are deseribed and traetable. 
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APPENDIX B. AN EXAMPLE APPLICATION OE SEVER 


A company wants to provide an efficient lighting system for office buildings. 
From a systems perspective, “providing and efficient lighting system” consists of more 
than lighting a building. At a minimum, it consists of: 

planning efforts —^physical work required to properly design, test, integrate, and 
support the system to meet customer expectations 

development efforts —^physical work to research concepts, communieate ideas, 
proeure hardware, validate design, and integrate the system 

installation efforts —^physical work required to package the system, ship the 
system, install the system, and test the system 

operation effort —daily actions and costs by the eonsumer to operate the system 

maintenance and support efforts —eosts and time assoeiated with maintaining the 
system, preventing system failures, responding to system failures, and ensuring the 
system meets the customer’s daily expectations 

disposal efforts —^removal costs, environmental impaets, time to dispose, 
resourees required to dispose 

Can this eompany set themselves up for success? The answer to this question can 
be approaehed by applying SEVER. Eirst a funetional deeomposition of this business is 
performed. To simplify the example, one partieular funetion in the “development efforts” 
phase, El, “Illuminate workspaee” is eonsidered. Eor eompleteness, the reader should 
note that the sum of eaeh funetion’s SEVER-caleulated value is the total system value. 
Thus, the following process ean be carried out for each function in the funetional 
deeomposition, and a total system value ean be caleulated. The value of the first funetion, 
El, is stated, algebraically, as: 
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(B.l) 


y, 


IFl 


P 



Q P 

p * t 


The three grouped terms, — reflect realistic quantifications of 

V p t 

/t 

performance, quality, and investment. P, Q, and I must be determined to properly 
calculate the value for function, Fi. If P is considered ‘lumens’, $/t can be considered ‘$ 
paid/hour of operation’. Q/P can then be considered lost due to poor quality [during 
operation]/lumen’. Lastly, P/t can be defined as ‘Total # lumens operated/total hours of 
operation’. The most difficult term to quantify is the quality term. This term is based on 
Taguchi’s quality—a loss to society resulting from less-than-optimal quality. In this case, 
the poor quality can be represented by: lighting a building without occupants, non- 
optimal brightness, flickering lights, and unreliable illumination, to name a few. The 
effects on society from these situations are many. A few short-term loss examples include 
less than optimal working conditions, disgruntled employees, and wasted electricity. A 
few long term examples include an unmotivated workforce and employees requiring 
vision insurance. The most important thing to note in this example is the effect of poor 
quality, as defined by Taguchi, at some point, affects society as a whole (Taguchi, 1990). 
Certainly, a preliminary analysis not considering quality would not have considered a 
vision insurance company related to the performance of a lighting system. 


Now, the equation reads: 


' $ operated 


lumens 


'hour of operation 


'' $ lost due to poor quality ) ( Total # lumens operated ^ 


lumen 


- • 


total hrs of operation 


(B.2) 


The value of this function is represented in units of lumens. This example, 
although not a complete technical breakdown of SEVER, illustrates an important feature 
of SEVER—the ability to plan and model, from a bottom-up approach, while being able 
to ultimately execute the business, or strategy, in a top-down manner. 
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APPENDIX C. SERVER APPLIED TO TEMPO EUNCTIONAL 

DECOMPOSITION 


The first level of the funetional deeomposition (after mueh iteration) is shown in 
Figure 30. The “home plate” symbol represents further deeomposition not refieeted in the 
speeifie figure. Figures 31-34 represent the further deeomposition of funetion 1.0 
Analyze Situation. Figure 35 represents the further deeomposition of funetion 2.0 Play 
Game. 


Questions that were eonsidered during funetional deeomposition; 

• During gameplay, is it better to eonsider operating total utils independent 
of eomputer’s number of utils? 

• Should the user break down their deeision making to eoneentrate on one 
weapon type at a time? 

• Should the user eonsider ‘balaneing’ the portfolio of weapon ehoiees or 
operate one type in partieular? 

• Is there a differenee between offensive and defensive Utils? 

• Does purehasing Intel provide an advantage? 

• Does purehasing eounter-intel provide an advantage? 

• Eaeh weapon type differs in performanee (i.e., some offer more utils than 
others). Eaeh weapon type also differs in investment required (i.e., some 
are eheaper than others). Therefore, the value of eaeh is weapon system is 
different. Is value dependent on total amount available to spend? 

• Is the goal to distinguish between eompeting weapons of the same type 
(i.e., OAl vs. OA2 vs. OA3) or eompeting weapon types themselves (OA 
vs. OB, ete.) or both? 
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Figure 30. Top Level Functional Decomposition 
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Figure 31. Functional Decomposition for 'Gather Data' Function 
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Figure 32. Functional Decomposition for 'Breakdown Data' Function 
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Figure 33. Functional Decomposition for 'Develop Information' Function 
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Figure 34. Functional Decomposition for "Evaluate Information" Function 
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Figure 35. Functional Decomposition for "Play Game" Function 
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Table 2 describes function definitions. Only lowest level functions are defined. 
Higher level functions are defined by implementing their respective lower-level 
functions. 


Function ID 

1.0 Analyze Situation 

1.1 Gather Data 

1.1.1 Record Own Notes 

1.1.1.1 List Acquirable Weapon 

Systems 

1.1.1.2 
be 

List Weapon Systems that can 
Operated 

1.1.1.3 

List Environmental Parameters 

1.1.2 

Record Screen Opponent 

1.1.2.1 

List Offensive Threat Levels 

1.1.2.2 

List Defensive Threat Levels 

1.2 Breakdown Data 

1.2.1 

Formulate Rules 

1.2.1.1 

Define Elements 

1.2.1.2 

Define Relationships 

1.2.2 

Prioritize Rules 

1.2.2.1 

Determine Rule Importance 

1.2.2.2 Compare to Other Rules 

1.2.2.3 

Rank Rules by Importance 

1.2.3 

Sequence Rules 

1.3 Develop Information 

1.3.1 

Calculate 

1.3.1.1 

Populate Rules 

1.3.1.2 Apply Rules 

1.3.1.3 

Collect Rule Results 

1.3.2 

Relate Turn-to-turn 


Table 2. Function Descriptions 


Description 



The act of recording, in some physicai manner, aii 
avaiiabie weapon systems that can be bought for 
turn n 


The act of recording, in some physicai manner, aii 
weapon systems, and the quantities avaiiabie, that 
are currently being stored in inventory to operate 
in turn n+1 


The act of recording, in some physical manner, 
probability of war and budget for turn n 


If offensive Intel is purchased during turn n-1, the 
act of recording offensive Intel results for net utils 


If defensive Intel is purchased during turn n-1, the 
act of recording defensive Intel results for net utils 



List all possible variables within the TEMPO 
model that can be represented 


List constraining interconnections between 
variables (i.e., functions/equations using above 
defined variables) 


Evaluate possible effects of rule on model outputs 
before each turn and calculate relative weights 
using pairwise comparison method 


Ensure complete set of rules exist 


Collect all rules together with weightings and sort 
in order of rank 


Arrange rules in the order that allows the specific 
algorithm to be implemented 



Insert applicable variables into given rules_ 


Determine rule outputs with given variable inputs 


Gather rule outputs to prepare for evaluation and 
information generation_ 
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Function ID 

1.4 Collect Historical Game Data 

1.5 Evaluate Information 

1.5.1 

Compare Rule Results 

1.5.1 

1 Compare to Historical Data 

1.5.1.2 Compare to Intel Data 

1.5.2 

Calculate Algorithm Results 

2.0 Play Game 

2.1 Provide Defense 

2.1.1 

Buy 

2.1.2 

Don’t Buy 

2.1.3 

Operate 

2.1.4 

Don’t Operate 

1 2.2 Provide Offense 

2.2.1 

Buy 

2.2.2 

Don’t Buy 

2.2.3 

Operate 


Don’t Operate 

1 2.3 Provide ‘Buy’ List 

2.3.1 

Defense 

2.3.2 

Offense 

2.3.3 

Defensive Intel 


Offensive Intel 

2.3.5 

Counter-Intel 

2.4 Provide ‘Operate’ List 

2.4.1 

Defensive 


Offensive 


Description_ 


Gather data from environmental and performance 
files from previous game outcomes and collate 
into organized inputs for comparison_ 


If Historical game data is collected (Function 1.4), 
judge performance of each rule against historical 
data 


If Offensive or Defensive Intel (Function 2.3.3 or 
2.3.4) is purchased, judge performance of each 
rule against corresponding Intel data_ 


Provide input for 2.3 based on outputs from 
1.3.1.3 



Select defensive weapon systems to buy based 
on output from Function 2.3.1. 


Bypass purchasing this defensive weapon system 


Select defensive weapon system to operate based 
on output from Function 2.4.1. 


Bypass operating this defensive weapon system 


Select offensive weapon systems to buy based on 
output from Function 2.3.2. 


Bypass purchasing this offensive weapon system 


Select offensive weapon system to operate based 
on output from 2.4.2 


Bypass operating this offensive weapon system 


Based on output from Function 1.5.2, list type and 
quantity of defensive weapon systems to acquire 


Based on output from Function 1.5.2, list type and 
quantity of offensive weapon systems to acquire 


Based on output from Function 1.5.2, display 
whether defensive Intel shall be purchased_ 


Based on output from Function 1.5.2, display 
whether offensive Intel shall be purchased_ 


Based on output from Function 1.5.2, display 
whether counter-Intel shall be purchased_ 


Based on output from Function 1.5.2, list type and 
quantity of defensive weapon systems to operate 


Based on output from Function 1.5.2, list type and 
quantity of offensive weapon systems to operate 
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APPENDIX D. VERIFICATION MATRICES FOR EACH 

STRATEGY 


Verification of each strategy was executed once the first version of each strategy 
was coded. It involved decomposing the TEMPO game into multiple worst-case 
scenarios and verifying the strategies worked under each. The log tables below display 
which strategies worked in which situations. Each strategy not using intelligence is 
verified by 16 figures—one for each extreme situation described in Table 3. Currently, 
only one strategy was implemented to use the Intelligence reported by TEMPO. This 
strategy is verified by 32 figures—one for each extreme situation described in Table 4. 
All figures are presented within the ‘Strategy Verification’ on the CD containing the 
entire TEMPO Version 3 software package to demonstrate that strategy verification was 
performed. 


Table 3. Tested Categories for all Strategies Except ‘Intel’ 


Weapons 

available 

Max Budget 
Available 

No Budget 
Available 

All 

0 

0 

None 

0 

0 

OA 

0 

0 

OB 

0 

0 

Offensive 

0 

0 

DA 

0 

0 

DB 

0 

0 

Defensive 

0 

0 
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Table 4. Tested Seenarios for 'Intel' Strategy 


Weapons 

available 

Max Budget Available 

No Budget Available 


Enemy 

Purchased... 

No Intel 

Enemy 

Purchased... 

No Intel 

All 

0 

0 

0 

0 

None 

0 

0 

0 

0 

OA 

0 

0 

0 

0 

OB 

0 

0 

0 

0 

Offensive 

0 

0 

0 

0 

DA 

0 

0 

0 

0 

DB 

0 

0 

0 

0 

Defensive 

0 

0 

0 

0 
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APPENDIX E. SOETWARE PACKAGE INSTALLATION 

INSTRUCTIONS 


For first-time installation; 

1. Download file and rename from; ‘TEMPOSetup.exx’ to; 
‘TEMPO_Setup.exe’. This file is 143 MB in size, so it eould take some 
time depending on your internet eonneetion speed. 

a. Double-click ‘TEMPO_Setup.exe’ 

b. Select ‘Yes’ to install the MATEAB Component Run-time (see 
Eigure 36). 

NOTE; This installation will take from 2 minutes to 10 minutes, depending on 
computer. 



Eigure 36. Install MATEAB Component Runtime (MCR) Screenshot 


1. The MCR installer will extract (MCRInstaller.exe). You will then be 
prompted to select a language. Continue to install the MATEAB 
Component Runtime (see Eigure 37). 
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Figure 37. MATLAB Component Runtime Installation Screenshot 


2. Enter appropriate name and organization for your computer. This 
irrelevant to TEMPO operations (see Eigure 38). 


MATLAB Component Runtime 7.S - InstdNShield Wizard 


Customer Information 
Ptease enter your information. 

User Name: 

|Any Name 

Organization: 

|Any Organization 



Instal this appkation for: 

f* Anyone who uses this computer (al users) 
C Only for me (NAVSEAWC NEWPORT) 



Eigure 38. MCR Installation Data Required Screenshot 
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3. 


Continue installation to the default direetory (i.e., ‘CAProgram Files\...’) 
(Figure 39). 



Figure 39. MCR Installation Complete Sereenshot 


a. Click ‘Yes’ to install Microsoft .NET Framework 3.0 Service Pack 
1. Note; By clicking this link you will be directed to a Microsoft 
network location (see Figure 40). 



Figure 40. Install Microsoft’s .NET 3.0 Eramework Screenshot 
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b. Accept the terms of the license agreement. 

2. The installation package will then begin downloading. The total download 
size is 58 MB (see Figure 41). 



Figure 41. Microsoft's .NET 3.0 Framework... Downloading... Screenshot 

a. The installation will automatically occur and complete (see Figure 
42). Visit ‘Windows Update’ if you desire to download the latest 
serviee packs and security updates. 
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1^ Microsoft .NET Framework 3.0 SPI Setup 


Setup Complete 


.not Framework 


Microsoft JCT Framework 3.0 SPI has been instaHed successFuUy. 

V It K highiy reconwnerxled that you download and instal the latest servKe packs and security 
updates For this product. 

For more nformation, see 



Figure 42. Microsoft's .NET 3.0 Framework Installation Complete Screenshot 


b. Click ‘Exit’ and the TEMPO.NET installation will continue and 
complete (see Figure 43). 



Figure 43. TEMPO.NET Installation Complete Screenshot 
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To run/play TEMPO; 

1. To run TEMPO.NET navigate to ‘Start’ ‘All Programs’ ‘NPS 
TEMPO’ (see Eigure 44). 



Eigure 44. Run TEMPO Sereenshot 


2. The ‘Start a New Game’ Window will appear (see Eigure 45). If at any 
point this window is not visible, click on ‘Eile’ ‘New Game’. If you are 
unable to do so, the window is hidden. To unhide, while focused on the 
TEMPO.NET Application, hold the ‘Alt’ key and continue to press ‘Tab’ 

until the icon is highlighted. When released, the ‘Start a New Game’ 
window will reappear. 
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Figure 45. Game Startup Screenshot 


3. At this point, the choice to play a ‘Manual’ Game (human vs. computer) 
or an ‘Automatic’ Game (strategy vs. computer) is presented. Both screens 
are shown below in Figure 46. 




Figure 46. 'Start a New Game' Options Screenshots 
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4. 


For a ‘Manual’ game, skip to step 5. For an ‘Automatie’ game, first enter 
the number of desired trials. Then, seleet a strategy. In this version, the 
user ean seleet any strategy, however, only the first six strategies will 
work. Other strategies will eause the applieation to hang up. Please only 
ehoose one of the first 6 to use. 

NOTE; Eaeh trial, takes anywhere between 10 seeonds and 2 minutes, depending 
on the number of turns exeeuted before war oeeurs. Please eonsider this timing when 
ehoosing the number of trials! 

5. Onee a seleetion is made, eliek ‘Play’ to begin! 

NOTE; Please be patient. It does take some time for the first run to initialize and 
exeeute after a fresh installation—sometimes as long as 2 minutes. Initialization of the 
MATLAB eomponent runtime eode eauses this signifieant delay. 

6. After play is eompleted, the assoeiated log file ean be viewed. To do so, 
navigate to ‘C;\TEMPO\logs’. All applieable log files are stored here, in 
XME format. 

To uninstall TEMPO; 

WARNING; The Uninstallation proeess will remove AEE TEMPO.NET log files 
loeated within the ‘C;\TEMPO’ direetory. If this is undesired, move these files to a 
separate folder outside of the ‘C;\TEMPO’ direetory strueture! 

1. There are two methods to uninstall TEMPO.NET. The first method is to 
navigate to ‘Start’ ‘All Programs’ ‘NPS TEMPO’ and eliek 
‘Uninstall TEMPO’. The seeond method is to navigate to ‘C;\TEMPO’ 
and eliek the ‘Uninstall TEMPO’ ieon. Both will initialize the uninstaller. 

a. The user should seleet whether to remove the MATEAB 
eomponent runtime from the maehine. Do this only if you do not 
plan to run TEMPO.NET again on the maehine. 

b. If the TEMPO.NET applieation has not been played after the 
installation, the removal will be eomplete (i.e., all assoeiated files 
and direetories will automatieally be removed). However, if 
TEMPO.NET has been exeeuted at least onee, the user must 
manually delete the ‘C;\TEMPO’ direetory. This will be apparent 
if a message box appears during the removal proeess (see Eigure 
47). 
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Figure 47. TEMPO.NET Directory Removal Screenshot 


c. After deleting ‘CATEMPO’ the TEMPO.NET application removal 
is complete! 


109 






















THIS PAGE INTENTIONALLY LEET BLANK 


no 



LIST OF REFERENCES 


Abraham, A. Jain, L. C., & Goldberg, R. (2005). Evolutionary multiobjective 

optimization: Theoretical advances and applications. London: Springer-Verlag. 

A1 Mannai, W. I. & Lewis, T.G. (2007). Minimizing network risk with application to 
critical infrastructure protection. Journal of Information Warfare, (5(2), 52-68. 

Baeck, T., Logel, D.B., & Z. Michalewicz, Z. (2000).Evolutionary computation: basic 
algorithms and operators. Boca Raton, LL: CRC Press. 

Bernoulli, D. (1954). Exposition of a new theory on the measurement of risk. (L. 

Sommer, Trans). Econometrica, 22(1), 22-36. (doi: 10.2307/1909829.) (Original 
work published 1738). Retrieved May 30, 2006, from 
http://www.math.fau.edu/richman/Ideas/daniel.htm 

Chapman, G. B. & Johnson, E. J. (1995). Preference reversals in monetary and life 
expectancy evaluations. Organizational Behavior and Human Decision 
Processes, 62, 300-317. 

DoD Handbook 4245.8. “Value Engineering.” (1986). Authorized by DoD Directive 
4245.8. Arlington, VA: Defense Technical Information Center. 

Doob, J.L. (1971). What is a martingale? The American Mathematical Monthly, 78{5), 
461^73. 

Eederal Aviation Administration. (2006). Trade studies section 4.6 in System Engineering 
Manual Version 3.1. Retrieved May 30, 2006, from 

http://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/opera 
tions/sysengsaf/seman/SEM3.1/Ch_l .pdf 

field, K.A. (1999, July), fast track to management. Design News. 

Game Theory. Pareto efficient. Retrieved May 30, 2006, from 

http://www.gametheory.net/dictionary/ParetoEfficient.html 

Howard, R. A. (1960). Dynamic programming and Markov processes. Cambridge, MA: 
MIT Press. 

Hubertus, T. J., Meer, K., & Triesch, E. (2004). Optimization theory. New York: 

Springer. 


Ill 



Johnson, R.W., et al. (2005). Coevolutionary optimization of fuzzy logic intelligence for 
strategic decision support. IEEE Transactions on Evolutionary Computation, 9(6), 
682-694. 

Keeney, R. L., & Raiffa, H. (1976). Decisions with multiple objectives. Cambridge, 
England: Cambridge University Press. 

Krantz, D. H., and Kunreuther, H.C. (2007). Goals and plans in decision making. 
Judgment and Decision Making, 2 (3), 137-168. 

Lakatos, I. (1978). The Methodology of Scientific Research Programmes: Volume 1: 
Philosophical Papers. Cambridge, England: Cambridge University Press. 

Langford, G., & Huynh, T. (2007, September). A methodology for managing complexity, 
systems engineering test and evaluation. Presented at Complex Systems and 
Sustainability Conference, Sydney, Australia. 

Langford, G.O. (2007). Reducing risk in designing new products using rapid Systems 
Engineering Proceedings, Asia-Pacific Systems Engineering Conference, paper 
no. 18. Singapore. 

Langford, G.O. (2006). Reducing risk of new business start-ups using rapid Systems 
Engineering. Proceedings of the Eourth Annual conference on Systems 
Engineering Research, paper no. 140, Los Angeles, CA. 

Langford, G.O., Lranck, R., Huynh, T., & Lewis, L, (2007). Gap analysis: rethinking the 
conceptual foundations. (Report No. NPS-AM-07-051). Monterey, CA: Naval 
Postgraduate School. 

Langford, G. O. (2007, March). Reducing Risk in Designing New Products Using Rapid 
Systems Engineering. Paper presented at Asia-Pacific Systems Engineering 
Conference, Singapore. 

Lewis, T. (2006). Critical infrastructure protection in homeland security. Hoboken, NJ: 
John Wiley & Sons. 

Lowrance, W. W. (1976). Of acceptable risk. Los Altos, CA: William Kaufman, Inc. 

Management Concepts. (2003). Defense acquisition guidebook. (Republished in 2008). 
Vienna, VA: Management Concepts Inc. 

Miles, L. D. (1972). Techniques for value analysis and engineering (2nd ed.). New York: 
McGraw Hill. 


112 



Resnik, M. (1987). Choices: An introduction to decision theory. Minneapolis, MN; 
University of Minnesota Press. 

Saaty, T.L. (2001). The analytic network process: decision making with dependence and 
feedback. Pittsburgh, PA; RWS Publieations. 

Saaty, T.L. (2001). Decision making for leaders: the analytic hierarchy process for 
decisions in a complex world. Pittsburgh, PA; RWS Publieations. 

Slovie, P. (1995). The eonstruetion of preferenee. American Psychologist, 50, 364-371. 

Steuer, R.E. (1986). Multiple criteria optimization: Theory, computations, and 
application. New York; John Wiley & Sons, Ine. 

Sutton, R. S. & Barto, A. G. (1998). Reinforcement learning: An introduction. 
Cambridge, MA; MIT Press. 

Taguehi, G. & Chowdhury, S., & Wu, Y. (2005). Taguchi’s quality engineering 
handbook. Hoboken, NJ; Wiley-lnterseienee. 

Taguehi, G. (1990). Introduction to quality engineering. Tokyo, Japan; Asian 
Produetivity Organization. 

TEMPO military planning game, explanation and rules for players, (n.d.). Internal 
doeument. Naval Postgraduate Sehool, CA. 

Tversky, A., Sattath, S. & Slovie, P. (1988). Contingent weighting and judgment and 
ehoiee. Psychological Review, 95, 371-384. 

Tversky, A., Slovie, P. & Kahneman, D. (1988). The eauses of preferenee reversal. 
American Economic Review, 80, 204-217. 

Zadeh, E. (1965). Euzzy Sets. Information and Control, 8, 338-353. 


113 



THIS PAGE INTENTIONALLY LEET BLANK 


II4 



INITIAL DISTRIBUTION LIST 


1. Defense Teehnieal Information Center 
Ft. Belvoir, Virginia 

2. Dudley Knox Library 
Naval Postgraduate Sehool 
Monterey, California 

3. Gary O. Langford 

Naval Postgraduate Sehool 
Monterey, California 

4. Rodney W. Johnson 
Naval Postgraduate Sehool 
Monterey, California 

5. Robert J. Chaves 

Naval Undersea Warfare Center, Division Newport 
Newport, Rhode Island 

6. Mark Rodrigues 

Naval Undersea Warfare Center, Division Newport 
Newport, Rhode Island 


115 



